To study the demographics of interstellar ices in the Central Molecular Zone (CMZ) of the Milky Way, we obtain near-infrared spectra of 109 red point sources using NASA IRTF/SpeX at Maunakea. We select the sample from near- and mid-infrared photometry, including 12 objects in the previous paper of this series, to ensure that these sources trace a large amount of absorption through clouds in each line of sight. We find that most of the sample (100 objects) show CO band-head absorption at 2.3 μm, tagging them as red (super-) giants. Despite the photospheric signature, however, a fraction of the sample with L-band spectra (9/82 = 0.11) exhibit large H2O ice column densities (N > 2 × 1018 cm−2), and six of them also reveal CH3OH ice absorption. As one of such objects is identified as a young stellar object (YSO) in our previous work, these ice-rich sight lines are likely associated with background stars in projection to an extended envelope of a YSO or a dense cloud core. The low frequency of such objects in the early stage of stellar evolution implies a low star-formation rate (≲0.02 M ⊙ yr−1), reinforcing the previous claim on the suppressed star-formation activity in the CMZ. Our data also indicate that the strong “shoulder” CO2 ice absorption at 15.4 μm observed in YSO candidates in the previous paper arises from CH3OH-rich ice grains having a large CO2 concentration [N(CO2)/N(CH3OH) ≈ 1/3].