It wont hurt the motor to remove it. Once you remove the oil cooler adapter you will need to get the correct oil filter spin on thingy to mount the filer.
Personally, I'd leave it on if you're not having problems with it. If you do take it off let me know what you want for it, I'd like to get it from you (I've been looking for one from mine).