Implicit or automatic detection of social signals, which discriminate animate, intentional objects in the environment, is essential for higher social cognition and its development. Using functional magnetic resonance imaging, we identified the neural substrate of detecting simple visual social signals and examined its functional link with the mechanism of inferring another's mental state. Healthy participants were presented with the eye-gaze shift (EG) and self-propelling motion (SP) under both implicit and explicit task conditions. They also performed a social role-playing game in which mental inference (MI) was implicitly prompted during the presentation of faces (implicit MI). Implicit detection of EG and SP activated the posterior middle temporal gyrus (pMTG) bilaterally, whereas the right posterior superior temporal sulcus was activated during the explicit conditions. We revealed that the individual variation in neural response in the right pMTG during implicit eye-gaze detection explains the individual tendency to recruit the regions implicated in mental-state inference (medial prefrontal cortex, temporal pole and striatum) during the implicit MI task. Our results suggest that the implicit detection of visual social signals involves the pMTG and underlies the development of higher social cognition.