Individuals with depression often experience widespread and persistent cognitive deficits, which might be due to brain atrophy and cerebral small vessel disease (CSVD). We therefore studied the associations between depression, markers of brain atrophy and CSVD, and cognitive functioning. We used cross-sectional data from the population-based Maastricht study (n = 4734; mean age 59.1 ± 8.6 years, 50.2% women), which focuses on type 2 diabetes. A current episode of major depressive disorder (MDD, n = 151) was assessed by the Mini-International Neuropsychiatric Interview. Volumes of cerebral spinal fluid, white matter, gray matter and white matter hyperintensities, presence of lacunar infarcts and cerebral microbleeds, and total CSVD burden were assessed by 3 T magnetic resonance imaging. Multiple linear and logistic regression analyses tested the associations between MDD, brain markers and cognitive functioning in memory, information processing speed, and executive functioning & attention, and presence of cognitive impairment. Structural equation modeling was used to test mediation. In fully adjusted models, MDD was associated with lower scores in information processing speed [mean difference = -0.18(-0.28;-0.08)], executive functioning & attention [mean difference = -0.13(-0.25;-0.02)], and with higher odds of cognitive impairment [odds ratio (OR) = 1.60(1.06;2.40)]. MDD was associated with CSVD in participants without type 2 diabetes [OR = 1.65(1.06;2.56)], but CSVD or other markers of brain atrophy or CSVD did not mediate the association with cognitive functioning. MDD is associated with more impaired information processing speed and executive functioning & attention, and overall cognitive impairment. Furthermore, MDD was associated with CSVD in participants without type 2 diabetes, but this association did not explain an impaired cognitive profile.