the tragedy lies in the fact that people aren't being kind to one another. "What I want to know is are you kind?"
I understand your situation and the guy being a dick, but you also spread negative energy after letting it build up. Of course situations are reactionary & you didn't want to have a confrontation, it just sucks that he lead you to that. What an idiot (he was). Not even saying I would've done different, it just sucks
As for OP's problem - I had similar experiences (but without children as I have none) in Hampton & Charleston. Some people think their seats don't mean anything and they're entitled to be where they want. Yet they want to go grab a beer or disgusting venue food or come back right at the climax or favorite part of a song instead of politely waiting until the song is over (like most do at most ALL musical events).
There are seat-borrows who gladly return them with a smile and "thanks", and just rude idiots ignorant of show ethics who talk during songs they either don't know or "can't dance to" - sounds like personal problems to me - take that shit outside in the lobby.
Please take your kids. My parents first took me in 94' when I was 6 myself (I don't believe in coincidence) & sure I got stepped on & sure we left early, but I've never looked back. Hell I still get stepped on. I don't leave early now though so keep it up & they'll surely come around in a circle