I've seen cases where a pilot bushing was just pounded in further,and the torque converter pilot was still able to fit ,but a few others the bushing had to be pulled out or the converter pilot would bottom out on it and not let the transmission & converter bolt up..
I would remove it if it were me..
It's easy to yank a brass pilot bushing out,just screw in a tap till it bottoms out and either keep turning,or take the tap out and screw a bolt in (if you fear the tap would break off)--usually they come right out without much force..the needle bearing type can be a pain,a cutting torch is the quickest way to get rid of that type--or you can use a thin cape point chisel to fold it inward till it can be extracted,if you lack the correct puller..