PASWT makes a good point about having the permit to hand over to the authorities in the event of an accident/incident.
However over the last few years I've driven in many other countries (including Australia) without the need to have an IDP. In any event the Isle of Man is not in the EU so it shouldn't make any difference if the UK leaves?
I guess its a case of the old "insurance adage". I'm sure that IOMG would welcome the revenue, if Brexit goes ahead what are the chances of the current rates going up!