Objective: The purpose of this article was to use the Random Forest method and nonlinear mixed-effects method to develop a model for determining tree height–diameter at breast height (DBH) for a natural coniferous and broad-leaved mixed forest in Jilin Province and to compare the advantages and disadvantages of the two methods to provide a basis for forest management practice. Method: Based on the Chinese national forest inventory data, the Random Forest method and nonlinear mixed-effects method were used to develop a tree height–DBH model for a natural coniferous and broad-leaved mixed forest in Jilin Province. Results: The Random Forest method performed well on both the fitting set and validation set, with an R2 of 0.970, MAE of 0.605, and RMSE of 0.796 for the fitting set and R2 of 0.801, MAE of 1.44 m, and RMSE of 1.881 m for the validation set. Compared with the nonlinear mixed-effects method, the Random Forest model improved R2 by 33.83%, while the MAE and RMSE decreased by 67.74% and 66.44%, respectively, in the fitting set; the Random Forest model improved R2 by 9.88%, while the MAE and RMSE decreased by 14.38% and 12.05%, respectively, in the validation set. Conclusions: The tree height–DBH model constructed based on the Random Forest method had higher prediction accuracy for a natural coniferous and broad-leaved mixed forest in Jilin Province and had stronger adaptability for higher-dimensional data, which can be used for tree height prediction in the study area.