Many football games close roads in vicinity, some golf tournaments require roads in area closed, parks commandeered for parking etc. I could go on, but those two are the sports I know most about so will stick on solid ground.
Yes the TT is fairly unique (apart from rallying, running, cycling) in that it closes roads for actual event use, rather than ancilliary use, but being against it because it closes roads is no argument at all.
As a one time comeover, I found the whole TT week fantastic, closed roads and all. But if you don't like it, let out your house for the week/fortnight, you'll more than pay for your foreign holiday. What's not to like.