Milk thistle extract that is widely supplied in the market has three main active ingredients, including silybin, silychristin and silydianin (SS), and isosilybin, they have a lot of valuable functions for the human body. The determination of these active ingredients is of great significance for customers, so it is urgent to develop convenient, fast, and on-site analytical methods. In this study, we developed a fast method to detect the active ingredients using a hand-held NIR spectrometer MicroNIR 1700 to record spectra and applied multiple pretreatment methods to process them. The random frog (RG) method was used to optimize wavelength variables with high information, and the partial least square (PLS) regression was used to calibrate the spectra data of silybin, SS, and isosilybin. The results showed that the standard normal variate transformation (SNV), SNV+ first derivative, and first derivative were the best pretreatment methods for silybin, SS, and isosilybin. The calibration performances of selected wavelength variables were significantly improved compared to the whole wavelength variables. The the minimum limit of quantification (LOQmin) of the silybin, SS, and isosilybin were 9.53%, 10.86%, and 2.64%, respectively. For the validation of new unknown samples, the root mean squared error of prediction (RMSEP) of silybin, SS, and isosilybin were 0.666%, 0.510%, and 0.157%, and the corresponding R2 were 0.992, 0.898, and 0.936, respectively, it demonstrated that the proposed methods are accurate and robust. Benefiting from the portable hand-held NIR spectrometer and the reliable algorithm, the present method offers a promising approach for the determination of the active ingredients in milk thistle extract because it is rapid, convenient, non-polluting, and on-site, does not require professional skills and expensive apparatus.