Abstract
Background: Clinical prediction models (CPMs) hold the potential to improve decision-making and individualize care for patients with cardiovascular disease (CVD); however, it is increasingly recognized that CPM performance may decrease when validated externally. Here we systematically describe published external validations of CPMs for patients with CVD, with specific attention to changes in CPM performance. Methods: A systematic review was conducted using a citation search to identify external validations of CPMs in the Tufts Predictive Analytics and Comparative Effectiveness (PACE) CPM Registry, a comprehensive database of CPMs for patients with CVD published between 1990 and May 2015. Information on CPM performance in the original derivation database as well as during subsequent external validations (through March 2017) was extracted. The percent change in discrimination was calculated as [(Validation AUC - 0.5) - (Derivation AUC - 0.5) / (Derivation AUC - 0.5) * 100]. Results: The Registry includes 1083 CPMs for CVD. 1555 external validations were identified. On average there were 1.4 validations/ de novo CPM (range 0-83). 758 (70%) of the CPMs have never been externally validated. The median external validation sample size was 1215 (IQR 430, 6509) and the median number of events 59 (21, 209). 88% (1368) of the external validations report area under the receiver operating characteristic curve (AUROC). 59% (917) report some measure of CPM calibration. The median external validation AUROC was 0.74 (0.69-0.81) and the median percent change in discrimination was -14.7% (-34.1%, +0.8%). 29.3% (455) of model validations showed CPM discrimination at or above the performance reported in the derivation dataset. Conclusion: While numerous CPMs exist for patients with CVD, most have never been externally validated. CPMs generally show substantially worse discrimination in external validations compared to the derivation datasets.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
