Immune checkpoint inhibitors (ICIs) have improved the long-term survival of NSCLC patients. However, the efficacy of ICIs in elderly NSCLC patients remains controversial. We conducted a retrospective study and meta-analysis exploring the efficacy of ICIs in those patients using public databases and RCTs. NSCLC patients were identified into elderly and non-elderly groups by age 75 years. The retrospective study showed significant differences in OS and PFS between non-elderly and elderly patients treated with ICIs (P= 0.029 and 0.027), with reduced efficacy in elderly NSCLC patients. ECOG PS also negatively affected OS in elderly NSCLC patients (P= 0.007). In meta-analysis, the HR for OS in the non-elderly and elderly groups were 0.74 and 0.90, respectively, and the difference between the two age groups was statistically significant (P= 0.025). ICIs resulted in a lower incidence of all-grade (OR= 0.47) and high-grade TRAEs (OR= 0.38) than chemotherapy. Our findings revealed that the survival benefit of ICIs in elderly patients (≥ 75 years) may be lower than in non-elderly patients. In addition, the incidence of TRAEs induced by ICIs was lower than chemotherapy.