The investigation of disease course-associated brain structural alterations in Major Depressive Disorder (MDD) have resulted in heterogeneous findings, possibly due to low reliability of single clinical variables used for defining disease course. The present study employed a principal component analysis (PCA) on multiple clinical variables to investigate effects of cumulative lifetime illness burden on brain structure in a large and heterogeneous sample of MDD patients. Gray matter volumes (GMV) was estimated in n = 681 MDD patients (mean age: 35.87 years; SD = 12.89; 66.6% female) using voxel-based-morphometry. Five clinical variables were included in a PCA to obtain components reflecting disease course to associate resulting components with GMVs. The PCA yielded two main components: Hospitalization reflected by patients' frequency and duration of inpatient treatment and Duration of Illness reflected by the frequency and duration of depressive episodes. Hospitalization revealed negative associations with bilateral dorsolateral prefrontal cortex (DLPFC) and left insula volumes. Duration of Illness showed significant negative associations with left hippocampus and right DLPFC volumes. Results in the DLPFC and hippocampus remained significant after additional control for depressive symptom severity, psychopharmacotherapy, psychiatric comorbidities, and remission status. This study shows that a more severe and chronic lifetime disease course in MDD is associated with reduced volume in brain regions relevant for executive and cognitive functions and emotion regulation in a large sample of patients representing the broad heterogeneity of MDD disease course. These findings were only partly influenced by other clinical characteristics (e.g., remission status, psychopharmacological treatment).