Yep, there is no vacuum shutoff valve on the coolant through the heater core. Also, with the cowl intake on the HVAC system, when you are moving there is air flowing whether the fan is on or not. Your best bet is to make sure it's set to the coldest setting so the air goes mostly around the heater core. But also, much of the sealing pieces (some foam) inside the heater box will let some air go by anyway since they are probably old and dry rotted. I made new seals for that from sealing strips at the hardware store. You could also do like got blazers and put a shutoff in your heater hose. If you swapped a big block make sure it has the intake/WP bypass hose as well if you do that.
Also, some of the models(maybe older?) did not have a fan off position. Only low/med/high. So if you don't have a fan off you might consider getting a different controller with a fan off position.