OK, that makes more sense.
You have a couple of potential problems with the starter. First, is the overloads. Actually the overloads are first and second.
First, the overloads that handled the original motor are too low for the new motor. The new motor is going to draw more current than the original one.
The second problem concerns what type of overloads you have. If they are the "heater" type, then you should be fine with just changing the heaters. If it is the newer electronic type, then you will have to loop one of the legs.
I doubt its the newer type, since it would have already tripped. The newer type sense a dead leg, so you have to take the output of one of the overloads, and apply it to the input of the third one.
Then use the output of the third one as one leg of the motor.
As to what fires the starter, despite the name and all the contacts, its basically just a relay. You need to look at the nameplate to find out what the coil calls for. I suspect its 240 volts.
If the original system had a neutral, then 240 volts should have been available, so you usually use the lower voltage for the coil. However, there are starters that use the full voltage. But, since its just a two leg coil, it should be simple.
There will be a set or two of auxiliary contacts that you wire the coil through. These contacts open when the overload trips, and kills the motor.
You should hear the relay pull in when you power up the system. If it does not, and the motor still runs, then you have something wired wrong.
I have a schematic for a three phase motor starter that I use on some projects. Let me find it and I will post it in a few minutes.