Chlorophyll fluorescence (ChlF) parameters offer valuable insights into quantifying energy transfer and allocation at the photosystem level. However, tracking their variation based on reflectance spectral information remains challenging for large-scale remote sensing applications and ecological modeling. Spectral preprocessing methods, such as fractional-order derivatives (FODs), have been demonstrated to have advantages in highlighting spectral features. In this study, we developed and assessed the ability of novel spectral indices derived from FOD spectra and other spectral transformations to retrieve the ChlF parameters of various species and leaf groups. The results obtained showed that the empirical spectral indices were of low reliability in estimating the ChlF parameters. In contrast, the indices developed from low-order FOD spectra demonstrated a significant improvement in estimation. Furthermore, the incorporation of species specificity enhanced the tracking of the non-photochemical quenching (NPQ) of sunlit leaves (R2 = 0.61, r = 0.79, RMSE = 0.15, MAE = 0.13), the fraction of PSII open centers (qL) of shaded leaves (R2 = 0.50, r = 0.71, RMSE = 0.09, MAE = 0.08), and the fluorescence quantum yield (ΦF) of shaded leaves (R2 = 0.71, r = 0.85, RMSE = 0.002, MAE = 0.001). Our study demonstrates the potential of FOD spectra in capturing variations in ChlF parameters. Nevertheless, given the complexity and sensitivity of ChlF parameters, it is prudent to exercise caution when utilizing spectral indices for tracking them.