This study aimed to accurately grasp the impact mechanism and change rule of buildings and green spaces on land surface temperature (LST), which is of great significance for alleviating urban heat islands (UHIs) and formulating adaptation measures. Taking Jinan, China, as the study area, combined multisource remote sensing data were used in this study to construct an index system of the influencing factors. We used a spatial regression model to explore the relative contribution of the influencing indicators on LST. We also drew a marginal utility curve to quantify the heating/cooling effect of the leading indicators. The results showed that, firstly, among the 3D building indicators, the leading indicators affecting LST were the degree of spatial convergence (SCD) and the building surface area (BSA). Among the green space indicators, the largest patch index (LPI), green coverage rate (GCR), and edge density (ED) were significantly negatively correlated with LST. Secondly, when we considered the 15 indicators comprehensively, SCD was the most influential indicator, with a contribution of 24.7%, and the contribution of the green space indicators to LST was significantly reduced. Thirdly, among the leading indicators, SCD was positively correlated with LST. When SCD was less than 60%, LST increased by about 0.38 °C for every 10% increase. When GCR > 44%, LST was significantly reduced, and when GCR > 62%, a cooling effect of 1.1 °C was observed. Beyond this threshold, the cooling effect will not improve significantly. This study shows that when 3D buildings are densely distributed and crowded, the cooling effect of green space will be limited to some extent by 3D buildings. The key to mitigating UHIs is to rationally configure and optimize the spatial structure of 3D buildings.