The composition and efficiency of heating, ventilation and air conditioning (HVAC) systems are crucial inputs for urban building energy modeling (UBEM). However, in current studies, the heterogeneity of HVAC systems is often ignored, leading to huge modeling errors. To address these issues, this study developed a three-step prediction approach that can dynamically forecast the composition of HVAC systems (SystemID), the efficiency of heat sources and the efficiency of entire systems at the urban scale. The performance of the prediction models was evaluated through the ten-fold cross-validation. The results showed that the accuracy in predicting the composition could reach 0.87, and the coefficient of determination (R2) was greater than 0.93 in efficiency forecasts. The accuracy of the developed approach was further evaluated through the testing set in two situations, i.e., with the predicted and actual SystemID data. In the first situation, the overall accuracy in predicting the composition reached 78.2 %, and the R2 for forecasting the system efficiencies was over 0.6. Then, with the SystemID assumed to be accurate, the R2 increased to more than 0.87. To analyze the performance of the developed method in energy use predictions, two case regions in Changzhou and Nanjing were used, respectively. The simulation results were compared with the scenario in which the system efficiencies were determined based on the national standard. The results showed that the developed method could enhance the accuracy of total energy use predictions by approximately 10 % and HVAC energy use predictions by roughly 40 % at both urban and building scales. This helps policy-makers craft energy-saving strategies more reasonably.