To address the challenge of quantitatively assessing the mentalization of emotions in color design schemes, this study uses Baidu Street View images and deep learning, integrates multi-source data, and innovatively constructs a color data model based on a comprehensive color indicator system for the quantitative assessment and visual representation of how the color environments of elementary school urban neighborhoods impact children’s mentalization of emotions. This model systematically incorporates physical color indicators, integrates elements such as perceptual frequency, and provides a novel perspective for color planning. The study’s results reveal that color metrics significantly impact children’s mentalization of emotions across multiple dimensions, with gender and age emerging as important influencing factors. Additionally, significant correlations were found between color and environmental elements such as building façades, roads, and signs. The study provides urban planners and architects with a practical color data model and recommendations for the revitalization of elementary school urban neighborhoods, offering a scientific basis for optimizing color design.