Don't hit the pinion with a hammer, you'll smash the threads on the pinion and you won't be able to get a new nut on there.
There is a little cup in the end of the pinion, put a punch in there and hit it with a hammer. As mentioned, the upper pinion bearing is pressed on there so it take a little doing to get it out.