As one of the world’s most popular fruit, apple tree yield prediction before harvest plays an important role in optimizing orchard nutrition management, especially at the individual tree level. However, few studies focus on fruit-tree yield prediction with remote-sensing technology whereas most of them aim at field crops. Current fruits identifying and counting methods often fail to produce the expected result due to light and occlusion in complex orchard conditions. Since both the spectral and morphological characteristics of tree canopy can reflect the growth and development of fruit trees and are directly related to its potential yield. In this study, we develop a channel for automatic extraction of spectral and morphological features of apple trees using light detection and ranging (LiDAR) and multispectral imagery data from unmanned aerial vehicles. The contribution of spectral and morphological characteristics to the yield prediction of individual apple trees is discussed. With the combination of spectral and morphological features, an ensemble machine learning yield prediction model was developed by combining two widely used basic learners: support vector regression (SVR) and K-nearest neighbor (KNN). Then through extrapolating the ensemble model, the yield map was produced at the orchard level and individual tree level, respectively. The results show that the data processing channels developed in this study can accurately extract the morphological and spectral features of individual apple trees. Three features (Crown Volume 1, Ratio Vegetation Index, and CPA1) contribute most in apple tree yield prediction. The ensemble learning model outperforms all base learners with R2 = 0.813 for the validation and 0.758 for the test when using the selected three features. This study thus provides a practical example of predicting the yield of individual apple trees based on multi-source remote-sensing data and ensemble learning.