A simple, experimental motor such as this is not capable of making much power. We can increase the turning force (or torque) that the motor can create in three ways: either we can have a more powerful permanent magnet, or we can increase the electric current flowing through the wire, or we can make the coil so it has many "turns" (loops) of very thin wire instead of one "turn" of thick wire. In practice, a motor also has the permanent magnet curved in a circular shape so it almost touches the coil of wire that rotates inside it. The closer together the magnet and the coil, the greater the force the motor can produce.
The first thing you will need to discover is whether you are dealing with a three-phase motor. You may already know this from the application, but another giveaway is that the lead wires of most three-phase motors are single colors, not multiple colors, and usually identified with numbers. If, on the other hand, the motor diameter is less than seven inches and has a terminal board, it is most likely a single-phase motor.
Without getting into a lengthy physics lecture, this process of reversing polarity produces heat (or wasted energy). This is known as hysteresis loss. And that helps explain why increasing the voltage into the motor will not necessarily increase the output. Instead, it can fight the resistance of magnetic materials to reverse polarity--and simply heat iron.
A ferrite sleeve around the three power conductors as they leave the drive will help to reduce common-mode noise current. Take all three conductors two or three times through the core. If it runs hot reduce the number of turns.