Near- and below-threshold harmonic generation provides a potential approach to generate vacuum-ultraviolet frequency comb. However, the dynamical origin of in these lower harmonics is less understood and largely unexplored. Here we perform an ab initio quantum study of the near- and below-threshold harmonic generation of caesium (Cs) atoms in an intense 3,600-nm mid-infrared laser field. Combining with a synchrosqueezing transform of the quantum time-frequency spectrum and an extended semiclassical analysis, the roles of multiphoton and multiple rescattering trajectories on the near- and below-threshold harmonic generation processes are clarified. We find that the multiphoton-dominated trajectories only involve the electrons scattered off the higher part of the combined atom-field potential followed by the absorption of many photons in near- and below-threshold regime. Furthermore, only the near-resonant below-threshold harmonic is exclusive to exhibit phase locked features. Our results shed light on the dynamic origin of the near- and below-threshold harmonic generation.