The production of high-quality tea by Camellia sinensis (L.) O. Ktze is the goal pursued by both producers and consumers. Rapid, nondestructive, and low-cost monitoring methods for monitoring tea quality could improve the tea quality and the economic benefits associated with tea. This research explored the possibility of monitoring tea leaf quality from multi-spectral images. Threshold segmentation and manual sampling methods were used to eliminate the image background, after which the spectral features were constructed. Based on this, the texture features of the multi-spectral images of the tea canopy were extracted. Three machine learning methods, partial least squares regression, support vector machine regression, and random forest regression (RFR), were used to construct and train multiple monitoring models. Further, the four key quality parameters of tea polyphenols, total sugars, free amino acids, and caffeine content were estimated using these models. Finally, the effects of automatic and manual image background removal methods, different regression methods, and texture features on the model accuracies were compared. The results showed that the spectral characteristics of the canopy of fresh tea leaves were significantly correlated with the tea quality parameters (r ≥ 0.462). Among the sampling methods, the EXG_Ostu sampling method was best for prediction, whereas, among the models, RFR was the best fitted modeling algorithm for three of four quality parameters. The R2 and root-mean-square error values of the built model were 0.85 and 0.16, respectively. In addition, the texture features extracted from the canopy image improved the prediction accuracy of most models. This research confirms the modeling application of a combination of multi-spectral images and chemometrics, as a low-cost, fast, reliable, and nondestructive quality control method, which can effectively monitor the quality of fresh tea leaves. This provides a scientific reference for the research and development of portable tea quality monitoring equipment that has general applicability in the future.