Patients with primary adrenal insufficiency (PAI) suffer from increased risk of infection, adrenal crises and have a higher mortality rate. Such dismal outcomes have been inferred to immune cell dysregulation because of unphysiological cortisol replacement. As the immune landscape of patients with different types of PAI has not been systematically explored, we set out to immunophenotype PAI patients with different causes of glucocorticoid (GC) deficiency. This cross-sectional single center study includes 28 patients with congenital adrenal hyperplasia (CAH), 27 after bilateral adrenalectomy due to Cushing's syndrome (BADx), 21 with Addison's disease (AD) and 52 healthy controls. All patients with PAI were on a stable GC replacement regimen with a median dose of 25 mg hydrocortisone per day. Peripheral blood mononuclear cells were isolated from heparinized blood samples. Immune cell subsets were analyzed using multicolor flow cytometry after four-hour stimulation with phorbol myristate acetate and ionomycin. Natural killer (NK-) cell cytotoxicity and clock gene expression were investigated. The percentage of T helper cell subsets was downregulated in AD patients (Th1 p = 0.0024, Th2 p = 0.0157, Th17 p < 0.0001) compared to controls. Cytotoxic T cell subsets were reduced in AD (Tc1 p = 0.0075, Tc2 p = 0.0154) and CAH patients (Tc1 p = 0.0055, Tc2 p = 0.0012) compared to controls. NKCC was reduced in all subsets of PAI patients, with smallest changes in CAH. Degranulation marker CD107a expression was upregulated in BADx and AD, not in CAH patients compared to controls (BADx p < 0.0001; AD p = 0.0002). In contrast to NK cell activating receptors, NK cell inhibiting receptor CD94 was upregulated in BADx and AD, but not in CAH patients (p < 0.0001). Although modulation in clock gene expression could be confirmed in our patient subgroups, major interindividual-intergroup dissimilarities were not detected. In patients with different etiologies of PAI, distinct differences in T and NK cell-phenotypes became apparent despite the use of same GC preparation and dose. Our results highlight unsuspected differences in immune cell composition and function in PAI patients of different causes and suggest disease-specific alterations that might necessitate disease-specific treatment.