I know a little bit of how you feel. I have a cabin about 4 hours away from here that was handed down to me from my grandpa. It is in ROUGH shape. The floors are rotting out and i think the whole north wall is sinking. The cabin is a massive part of my childhood and means a ton to me. But, it seems i never have the time/money to go up there and get work done. At this point, the thing is to the point where what really needs to happen is it needs demolished and a new one built. It will be many years before i can make that happen. So i thought, the cabin isnt in an ideal location, and it is an enormous undertaking to rebuild it which i just dont have time for. So maybe i should just get rid of it. But the more i thought about it, the more i realized, that i had two choices. I could get rid of it and never even have the chance of enjoying it again, or i could hold on to it and hold on to hope that i would someday be able to get it to a point where it can be enjoyable again. So i decided to keep it. This way, if it takes my 10 years until i can do anything with it, at least its not lost for a life time.
So its different than your situation, but kinda the same. And i think the right choice is to keep it and hope for the best. At least then you have a chance.