Hypophysitis is a known immune-related adverse event (irAE) of immune checkpoint inhibitors (CPIs), commonly associated with CTLA-4 inhibitors and less often with PD-1/PD-L1 inhibitors. We aimed to determine clinical, imaging, and HLA characteristics of CPI-induced hypophysitis (CPI-hypophysitis). We examined the clinical and biochemical characteristics, magnetic resonance imaging (MRI) of the pituitary, and association with HLA type in patients with CPI-hypophysitis. Forty-nine patients were identified. Mean age was 61.3 years, 61.2% were men, 81.6% were Caucasian, 38.8% had melanoma, and 44.5% received PD-1/PD-L1 inhibitor monotherapy while the remainder received CTLA-4 inhibitor monotherapy or CTLA-4/PD-1 inhibitor combination therapy. A comparison of CTLA-4 inhibitor exposure vs PD-1/PD-L1 inhibitor monotherapy revealed faster time to CPI-hypophysitis (median 84 vs 185 days, P < .01) and abnormal pituitary appearance on MRI (odds ratio 7.00, P = .03). We observed effect modification by sex in the association between CPI type and time to CPI-hypophysitis. In particular, anti-CTLA-4 exposed men had a shorter time to onset than women. MRI changes of the pituitary were most common at the time of hypophysitis diagnosis (55.6% enlarged, 37.0% normal, 7.4% empty or partially empty) but persisted in follow-up (23.8% enlarged, 57.1% normal, 19.1% empty or partially empty). HLA typing was done on 55 subjects; HLA type DQ0602 was over-represented in CPI-hypophysitis relative to the Caucasian American population (39.4% vs 21.5%, P = 0.01) and CPI population. The association of CPI-hypophysitis with HLA DQ0602 suggests a genetic risk for its development. The clinical phenotype of hypophysitis appears heterogenous, with differences in timing of onset, changes in thyroid function tests, MRI changes, and possibly sex related to CPI type. These factors may play an important role in our mechanistic understanding of CPI-hypophysitis.