Well, a man who scanned the periphery would come across as shifty (“what’s he looking for? is he some kind of voyeur or predator? he’s not staring at that girl’s tits, is he, the creep?”), so looking straight ahead is kind of like keeping one’s hands where everyone can see them. Though granted the absence of likely threats would also have an influence.