Abstract Background Cardiovascular risk- and screening scores guide clinical practice. Built upon simple risk factors, they could benefit from information on actual, subclinical cardiac changes. The ECG provides proxies for various cardiac states, which can be detected using machine learning. We systematically assess the added value of ECG-based cardiovascular risk factor estimates for six outcomes in two population-based cohorts: Study of Health in Pomerania (SHIP-Start) and UK Biobank (UKB). Methods We employed convolutional neural networks to analyze ECGs from the Hamburg City Health Study (HCHS) for the assessment of multiple risk factors, including age, weight, diabetes, atrial fibrillation (AF) and heart failure. Cox models were fitted on UKB and SHIP to predict 5 and 15 year time-to-event. Endpoints were death, cardiovascular death, AF, stroke, heart failure and myocardial infarction. Covariates were sex, age, body mass index, diabetes, hypertension and smoking (Cox-RF), ECG-predicted risk factors (Cox-AI), and a combination (Cox-RF/AI). Stratified Monte Carlo cross-validation was used as validation and groups of C-indices were compared using Mann-Whitney U tests. Results The study included 9103 HCHS participants (mean age 62.0±8.4 years, 51% women), 3546 SHIP participants (age 49.7±16.4, 51% women), and 32828 UKB participants (age 64±7.6 years, 52% women). Cox-AI performed comparable or better than Cox-RF for AF (SHIP: 0.88 vs. 0.84, UKB: 0.71 vs. 0.72) and heart failure (SHIP: 0.8 vs. 0.79, UKB: 0.76 vs. 0.75), but was less strong for the other outcomes. The combination of information on cardiovascular risk factors and ECG information (Cox-RF/AI) significantly improved the predictive ability compared to risk factors alone (Cox-RF) for AF (SHIP: 0.89 vs. 0.84, UKB: 0.76 vs. 0.72), cardiovascular death (SHIP: 0.90 vs. 0.89, UKB: 0.75 vs. 0.74) and heart failure (SHIP: 0.81 vs. 0.79, UKB: 0.80 vs. 0.75). Discussion ECG-based risk factors complement risk stratification by incorporating nuanced information on cardiac function and structure, improving risk prediction and thus possibly clinical decision making and patient outcomes.