Sexual and gender minority (SGM) groups have been historically underrepresented in neurologic research, and their brain health disparities are unknown. We aim to evaluate whether SGM persons are at higher risk of adverse brain health outcomes compared with cisgender straight (non-SGM) individuals. We conducted a cross-sectional study in the All of Us Research Program, a US population-based study, including all participants with information on gender identity and sexual orientation. We used baseline questionnaires to identify sexual minority (lesbian, gay, bisexual, diverse sexual orientation; nonstraight sexual orientation) and gender minority (gender diverse and transgender; gender identity different from sex assigned at birth) participants. The primary outcome was a composite of stroke, dementia, and late-life depression, assessed using electronic health record data and self-report. Secondarily, we evaluated each disease separately. Furthermore, we evaluated all subgroups of gender and sexual minorities stratified by sex assigned at birth. We used multivariable logistic regression (adjusted for age, sex assigned at birth, race/ethnicity, cardiovascular risk factors, other relevant comorbidities, and neighborhood deprivation index) to assess the relationship between SGM groups and the outcomes. Of 413,457 US adults enrolled between May 31, 2017, and June 30, 2022, we included 393,041 participants with available information on sexual orientation and gender identity (mean age 51 [SD 17] years), of whom 39,632 (10%) belonged to SGM groups. Of them, 38,528 (97%) belonged to a sexual minority and 4,431 (11%) to a gender minority. Compared with non-SGM, SGM persons had 15% higher odds of the brain health composite outcome (odds ratio [OR] 1.15, 95% CI 1.08-1.22). In secondary analyses, these results persisted across sexual and gender minorities separately (all 95% CIs > 1). Assessing individual diseases, all SGM groups had higher odds of dementia (SGM vs non-SGM: OR 1.14, 95% CI 1.00-1.29) and late-life depression (SGM vs non-SGM: OR 1.27, 95% CI 1.17-1.38) and transgender women had higher odds of stroke (OR 1.68, 95% CI 1.04-2.70). In a large US population study, SGM persons had higher odds of adverse brain health outcomes. Further research should explore structural causes of inequity to advance inclusive and diverse neurologic care.