It's a shame many (most?) people in our society are suspicious of strangers' intentions and are ready to jump to conclusions or 'have an angry attack,' but that's the society we have. It's generally not normal to approach strangers in public places and socially interact with them, as it might be in other (notably third world) countries.

I'd occasionally find myself shooting in populated public places, and although I know I'm perfectly within my legal rights to photograph strangers including children, I'd usually try to avoid singling out individuals or small groups to photograph. Also, I'd try to avoid appearing to do that. Why? Hardly from a fear of being called a paedophile, but mostly because it would feel rude. I feel like pointing a camera at a stranger is a bit like staring at them - if a stranger pointed a camera at me or my kids I'd feel like that isn't socially appropriate.

Actually, I reckon I'd be more accepting of a stranger pointing a DSLR at my kids than a phone camera. A DSLR suggests they are trying to create art, which I'm more likely to be fine with. A phone camera suggests they are trying to document the moment, which would make me question why.