Sporadic cerebral small vessel disease (CSVD) is a class of important pathologic processes known to affect the aging brain and to contribute to cognitive impairment. We aimed to identify clinical risk factors associated with postmortem CSVD in middle-aged to older adults. We developed and tested risk models for their predictive accuracy of a pathologic diagnosis of nonamyloid CSVD and cerebral amyloid angiopathy (CAA) in a retrospective sample of 160 autopsied cases from the Edinburgh Brain Bank. Individuals aged 40 years and older covering the spectrum of healthy aging and common forms of dementia (i.e., highly-prevalent etiologies such as Alzheimer disease (AD), vascular cognitive impairment (VCI), and mixed dementia) were included. We performed binomial logistic regression models using sample splitting and cross-validation methods. Demographics, lifestyle habits, traditional vascular risk factors, chronic medical conditions, APOE4, and cognitive status were assessed as potential predictors. Forty percent of our sample had a clinical diagnosis of dementia (AD = 33, VCI = 26 and mixed = 5) while others were cognitively healthy (n = 96). The mean age at death was 73.8 (SD 14.1) years, and 40% were female. The presence of none-to-mild vs moderate-to-severe nonamyloid CSVD was predicted by our model with good accuracy (area under the curve [AUC] = 0.84, sensitivity [SEN] = 72%, specificity [SPE] = 95%), with the most significant clinical predictors being age, history of cerebrovascular events, and cognitive impairment. The presence of CAA pathology was also predicted with high accuracy (AUC = 0.86, SEN = 93%, SPE = 79%). Significant predictors included alcohol intake, history of cerebrovascular events, and cognitive impairment. In a subset of atypical dementias (n = 24), our models provided poor predictive performance for both nonamyloid CSVD (AUC = 0.50) and CAA (AUC = 0.43). CSVD pathology can be predicted with high accuracy based on clinical factors in patients within the spectrum of AD, VCI, and normal aging. Whether this prediction can be enhanced by the addition of fluid and neuroimaging biomarkers warrants additional study. Improving our understanding of clinical determinants of vascular brain health may lead to novel strategies in the prevention and treatment of vascular etiologies contributing to cognitive decline. This study provides Class II evidence that selected clinical factors accurately distinguish between middle-aged to older adults with and without cerebrovascular small vessel disease (amyloid and nonamyloid) pathology.