I would say quit worrying, but.. .
The 700R4 and 350 transmissions have the governor right there and it get lubricated by oil as well. As long as the seal on the t-case input is good, there isn't a problem. Yes, put it together.
As far as the NV4500 having a seal there, that's a different trans, more splines, bigger shaft. And I would bet that there is still some oil that gets to the shaft, but I haven't had one of those split in a while. Can't remember for sure.
I can tell you what I have seen for sure with PTO splines that didn't get greased often enough, is a shortened life span. And the "oh crap! My truck doesn't work "
scenario.....