Urban environments have a strong influence on the land surface temperature (LST) in urban areas. Understanding the relationship between LST and urban environmental factors can help develop effective strategies to reduce high LSTs in urban areas, which is critical for mitigating the urban heat island effect. Previous studies have focused on the correlation between LST and the environmental factors that drive its formation, without considering the influences of the neighboring environment and the vertical expansion of highly urbanized areas. Notably, the correlation between LST and its neighboring environment in different seasons remains unclear. In this study, we selected central Beijing in China as our study area and employed the moving window method to characterize the environmental factors of the neighboring environment of the central LST cell. We explored eight environmental factors from three layers: normalized difference vegetation index (NDVI), normalized difference built-up index (NDBI), modified normalized difference water index (MNDWI), building density (BD), building height (BH), building volume (BV), sky view factor (SVF), and road density (RD). The Pearson correlation and extreme gradient boosting (XGB) regression methods were applied to measure the correlation between LST and the different factors in moving windows of different sizes. The results indicated that the correlation between NDVI, MNDWI, and LST was considerably different in the winter and other seasons. However, NDBI was positively correlated with LST in all seasons, although the correlation was strongest/weakest in summer/winter. Among building-related factors, BD and BH were more strongly correlated with LST, and the positive/negative correlation between BD/BH and LST was stronger in summer/winter. The correlation between LST and its neighboring environment varied with increasing window size, and this variation differs significantly between winter and other seasons. In spring, summer, and autumn, the strength of the correlation between LST and its neighboring environment showed an “inverted V” pattern with increasing window size. The optimal spatial scales to explore the influence of neighboring environments on the LST of 30-m cells were 210 m and 270 m. This study revealed the seasonal correlation between LST and its neighboring environment while explaining the variation at a spatial scale. Notably, this study can provide a new perspective for understanding the driving mechanism of the urban thermal environment, while contributing to its scientific optimization and management.