During the growth season, jujube trees are susceptible to infestation by the leaf mite, which reduces the fruit quality and productivity. Traditional monitoring techniques for mites are time-consuming, difficult, subjective, and result in a time lag. In this study, the method based on a particle swarm optimization (PSO) algorithm extreme learning machine for estimation of leaf chlorophyll content (SPAD) under leaf mite infestation in jujube was proposed. Initially, image data and SPAD values for jujube orchards under four severities of leaf mite infestation were collected for analysis. Six vegetation indices and SPAD value were chosen for correlation analysis to establish the estimation model for SPAD and the vegetation indices. To address the influence of colinearity between spectral bands, the feature band with the highest correlation coefficient was retrieved first using the successive projection algorithm. In the modeling process, the PSO correlation coefficient was initialized with the convergent optimal approximation of the fitness function value; the root mean square error (RMSE) of the predicted and measured values was derived as an indicator of PSO goodness-of-fit to solve the problems of ELM model weights, threshold randomness, and uncertainty of network parameters; and finally, an iterative update method was used to determine the particle fitness value to optimize the minimum error or iteration number. The results reflected that significant differences were observed in the spectral reflectance of the jujube canopy corresponding with the severity of leaf mite infestation, and the infestation severity was negatively correlated with the SPAD value of jujube leaves. The selected vegetation indices NDVI, RVI, PhRI, and MCARI were positively correlated with SPAD, whereas TCARI and GI were negatively correlated with SPAD. The accuracy of the optimized PSO-ELM model (R2 = 0.856, RMSE = 0.796) was superior to that of the ELM model alone (R2 = 0.748, RMSE = 1.689). The PSO-ELM model for remote sensing estimation of relative leaf chlorophyll content of jujube shows high fault tolerance and improved data-processing efficiency. The results provide a reference for the utility of UAV remote sensing for monitoring leaf mite infestation of jujube.