Immune checkpoint inhibitors (ICIs) have revolutionized cancer care with incredible reductions in mortality. One of the most devastating complications of treatment is ICI-related pneumonitis (ICI-p). Despite this, little is known regarding risk factors for severe pneumonitis and treatment effectiveness of various therapeutic options for steroid-refractory disease. To address this, we conducted a retrospective study on patients with cancer who developed ICI-p. We examined consecutive patients who received ICIs and developed ICI-p. Risk factors of interest for severe disease and steroid-refractory ICI-p, including pre-treatment pulmonary function tests (PFTs) and chest imaging, were compared between patients with severe (grades 3-5) and mild (grades 1-2) pneumonitis. The clinical and treatment courses for patients with steroid-refractory ICI-p were recorded. A total of 132 patients developed ICI-p, with 60 patients having mild and 72 with severe disease. We found that lower forced vital capacity percent predicted (66.24 vs 85.05, P = .05), lower total lung capacity percent predicted (85.23 vs 99.71, P = .13), and specific radiographic patterns on pre-treatment chest imaging were predictors of severe disease. Initial corticosteroid dose of less than 1 milligram per kilogram prednisone equivalent (P = .14) was correlated with partially steroid-responsive or steroid-refractory ICI-p. Ten patients had steroid refractory ICI-p, and those who received IVIG alone as the immune suppressant beyond corticosteroids had improved survival (P = 05). We are the first to identify pre-treatment PFTs and chest imaging abnormalities as risk factors for severe ICI-p. We also found that lower corticosteroid doses were associated with partially steroid-responsive and steroid-refractory ICI-p. Larger, prospective studies are needed to validate our results.