The smaller diameter is basically the inner race of the bearing. The finish rough finish should still be ok (like K85 said) since it sees relatively small movements - steering.
However, the larger diameter, closest to the flange, is a sealing surface and if you blasted it, it's gonna tear up your seal. Since there's grease in there and not oil, you might choose to live with it, but I would definitely expect to get a little more leakage when it gets hot and runny. Also be wary of water/mud and be sure to purge it with new grease if you wheel in the wet stuff.
Last thing. Hook your grease gun up to the zerx while it's all apart and purge the grease channels in all parts with said grease fittings. The sand will stick to the grease in there and as soon as you put it al together, the first time you go to lube everything up, you'll be pushing sand right into the works.