Numerosity perception is a fundamental cognitive function in humans and animals. Using an individual difference approach with a comprehensive dataset (N= 249), we performed a voxel-based morphometry analysis to unravel the neuroanatomical substrates associated with individual differences in numerosity perception sensitivity, measured by a classical non-symbolic numerical judgment task. Results showed that greater gray matter volume (GMV) in the left cerebellum, right temporal pole, and right parahippocampal was positively correlated to higher perceptual sensitivity to numerosity. In contrast, the GMV in the left intraparietal sulcus, and bilateral precentral/postcentral gyrus was negatively correlated to the sensitivity of numerosity perception. These findings indicate that a wide range of brain structures, rather than a specific anatomical structure or circuit, forms the neuroanatomical basis of numerosity perception, lending support to the emerging network view of the neural representation of numerosity. This work contributes to a more comprehensive understanding of how the brain processes numerical information.