Replacing driver side mirror unit.

[Thundergy]

So Fresh Direct truck clipped off driver side mirror on my 2002 BMW x5 4.4i while she was parked. Assholes said they're going to look into it but now they are not answering my calls or e-mails. Anyway i bought a whole new mirror unit and im wondering if i can replace the mirror without removing the door panel. But if its mandatory to remove the door panel, do you think it will be an easy-moderate DIY project, and approximately how long will this take. Again its driver side mirror, and i need to replace the whole thing, not just mirror panel.