The quest for GRACE 3.0: improving our beloved risk score with machine learning

Abstract

Abstract Background Although widely recommended for risk assessment of patients with acute coronary syndrome (ACS), the Global Registry of Acute Coronary Events (GRACE) score famously lacks discriminative power. On the other hand, in-hospital serum hemoglobin levels (HG) have been shown to simultaneously forecast both thrombotic and hemorrhagic hazards. Purpose To ascertain the extent to which the incorporation of HG in the GRACE score is able to increase its predictive ability. Methods Retrospective single-center study encompassing ACS patients consecutively admitted to a Cardiac Intensive Care Unit. Inclusion criteria comprised the acquaintance of GRACE score, HG and vital status on a 6-month follow-up, which served as the outcome. 3 discriminative models were first created: (standard) GRACE score (model 1); GRACE score plus HG, by means of logistic regression (model 2); GRACE score plus HG, by means of multilayer perceptron (a class of feedforward artificial neural network) (model 3). Hereafter, if models 2 and/or 3 were to be found significantly more discriminative than model 1, a correction factor would be calculated, also allowing for the conception of the most predictive model possible (model 4). The discriminative ability was estimated by both the area under the receiver-operating characteristic curve (AUC), and the dyad sensitivity/specificity. Results Between April 2009 and December 2016, 1468 patients met study inclusion criteria. Mean age was 68.0±13.2 years and 29.8% were female, while 36.9% presented with ST-segment elevation myocardial infarction. Mean GRACE score was 145.5±47.0 and mean HG was 13.5±2.0. All-cause mortality reached 10.5%, at 6 months. Predictive power for models 1, 2 and 3 may be quantified as follows: AUC 0.6998, sensitivity 77.7% and specificity 62.5%; AUC 0.7818, sensitivity 36.3% and specificity 92.2%; AUC 0.7851, sensitivity 47.7% and specificity 88.5%, respectively. Both models 2 and 3 exhibited more discriminative ability than model 1 (p<0.001), due to their higher specificity. As such, a correction factor was computed (y = −7.8556x + 86.4117) and model 4 was created, displaying a sensitivity of 65.9% and a specificity of 76.5%. Conclusion HG single-handedly provides incremental predictive value – namely more specificity – to the GRACE score. In particular, the latter seems to overestimate ACS patients' risk if HG is normal or close to normal. Funding Acknowledgement Type of funding sources: None.