Social interaction is critical to emotional well-being. Previous studies have suggested sex differences in the perception of social interaction. However, the findings depend on the nature of interactions and whether it involves facial emotions. Here, we explored sex differences in neural responses to the perception of social interaction using the Human Connectome Project data. Participants (n = 969, 505 women) were engaged in a social cognition task with geometric objects moving and colliding to simulate social interaction. Behaviorally, men relative to women demonstrated higher accuracy in perceiving social vs. random interactions. Men vs. women showed higher activation in the right superior temporal gyrus, bilateral occipital and posterior cingulate cortex and precuneus, and women vs. men showed higher activation in the right inferior frontal cortex, during exposure to social vs. random interactions. In whole-brain regressions, the differences in accuracy rate in identifying social vs. random interactions (ARSOC – ARRAN) were associated with higher activation in the paracentral lobule (PCL) and lower activation in bilateral anterior insula (AI), pre-supplementary motor area (preSMA), and left middle frontal gyrus (MFG) in men and women combined, lower activation in bilateral AI, preSMA and left MFG in men alone, and higher activation in the PCL and the medial orbitofrontal cortex in women alone. The latter sex differences were confirmed by slope tests. Further, the PCL activity mediated the correlation between an internalizing syndromal score, as assessed by the Achenbach Self-Report, and (ARSOC – ARRAN) across all subjects. These findings highlighted sex differences in the behavioral and neural processes underlying the perception of social interaction, as well as the influence of internalizing traits on these processes.