When I was young I used to put it down to just shyness but I now know there's more to it than that. It's like there is an undue pressure, an expectation beyond the norm and I almost feel like shouting "What? What do you want from me?"
At it's worst, rather than being a connection, it feels like an invasion, an imposition, another person almost usurping my attention even though the situation demands it.
Sometimes I can manage it, for a while at least, others I have to force myself but the effort is draining. Usually, I just have to find some reason, no matter how trivial, to break it: scratch an itch, pick at a bit of fluff, visually search for the source of a nearby noise, anything to reclaim the sovereignty of my sight.