Absolute vs Additive Net Reclassification Index

Abstract

_To the Editor_ Dr Alba and colleagues advocated for head-to-head comparison of clinical prediction models using a modification of the net reclassification index (NRI), termed the absolute NRI. The argument for the proposed modification to the commonly used form of the NRI, which they termed the additive NRI, is flawed. The authors claimed that “the major limitation of the additive NRI is that it does not consider the prevalence of the events and nonevents in the population.” They indicated that “… the results could be misleading” but that “the absolute NRI avoids this problem.” The additive NRI naturally weights the relative importance in improvement in risk classification for events and nonevents by the reciprocal of the odds of the event rate. Conversely, the absolute NRI completely disregards the event rate in the study population. Thereby, the absolute NRI does not avoid but creates problems by equally valuing improvements in sensitivity and specificity. This is only meaningful if clinical consequences are tied to a predicted risk threshold of exactly 50%, which will rarely be the case in practice. In preventive cardiology, such an approach would ascribe preference to missing an opportunity to treat 9 individuals who would have a cardiovascular event so that 10 individuals can avoid unnecessary treatment. In general, for serious health outcomes with an incidence well below 50%, improvements in sensitivity should be valued higher than improvements in specificity. For example, the 10-year cardiovascular disease risk threshold of 7.5% recommended by the American College of Cardiology/American Heart Association cholesterol treatment guidelines implies changes in sensitivity are valued 12.3 times as important as changes in specificity ([100-7.5]/7.5 = 12.3). Therefore, in the clinical scenario noted by Alba and colleagues, the recommendation for the addition of N-terminal pro-B-type natriuretic peptide to the Pooled Cohort Equations does not make sense solely based on a negative absolute NRI. Due to the equal weight assigned to events and nonevents, the absolute NRI is mathematically improper and has no clinical interpretation; therefore, it should not be used. We have argued that, when comparing risk prediction models, researchers should report the components of the NRI based on changes in risk classification of events and nonevents. This will enable fuller assessment of the effect on sensitivity and specificity and afford computation of decision-analytic summary measures such as the weighted NRI, relative utility, or net benefit, which can incorporate weighing factors appropriate for the situation at hand. In cases in which no meaningful risk thresholds exist, the NRI at event rate may offer a reasonable summary metric.