Another way of thinking about it is that the gears need to line up/not be under tension
Sometimes I bump it forward with the clutch and try again, or even back up and then push in the clutch and try.
If you are still fighting with it, you probably should investigate the shifter