Not a problem and legal so long as you have 2 mirrors.
1. Must have mirror on driver's side, and 2. either a rear view mirror or mirror on passenger side. I've been driving around w/out doors for a couple of months now, live across the street from a Sheriff's Station, and have not had a problem. I do, however, get a bunch of looks to see if I have a mirror attached on the driver's side, but mostly from Chippie's /forums/images/graemlins/thumb.gif
BTW: CA Vehicle Code Section 26709(a) establishes use of rear view mirror and at least left-hand mirror