Background: The prognostic nutritional index (PNI) and the systemic immune inflammation index (SII) have been used as simple risk-stratification predictors for COVID-19 severity and mortality in the general population. However, the associations between these indices and mortality might differ due to age-related changes such as inflammaging and several comorbid conditions in older patients. Therefore, we aimed to compare the predictivity of the PNI and SII for mortality among hospitalized older patients and patients under 65 years old. Methods: Patients hospitalized with COVID-19 from March 2020 to December 2020 were retrospectively included. The PNI and SII were calculated from hospital records within the first 48 h after admission. Data were evaluated in the whole group and according to age groups (≥65 < years). Receiver operating characteristic curves were drawn to evaluate the predictivity of the PNI and SII. Results: Out of 407 patients included in this study, 48.4% (n = 197) were older patients, and 51.6% (n = 210) were under 65 years old. For mortality, the area under the curve (AUC) of the PNI and SII in the adult group (<65 years) was 0.706 (95% CI 0.583–0.828) (p = 0.003) and 0.697 (95% CI 0.567–0.827) (p < 0.005), respectively. The AUC of the PNI and SII in the older group was 0.515 (95% CI 0.427–0.604) (p = 0.739) and 0.500 (95% CI 0.411–0.590) (p = 0.993). Conclusions: The accuracy of the PNI and SII in predicting mortality in adult COVID-19 patients seemed to be fair, but no association was found in geriatric patients in this study. The predictivity of the PNI and SII for mortality varies according to age groups.