Abstract
BackgroundSurrogate outcomes are often utilized when disease outcomes are difficult to directly measure. When a biological threshold effect exists, surrogate outcomes may only represent disease in specific subpopulations. We refer to these outcomes as “partial surrogate outcomes.” We hypothesized that risk models of partial surrogate outcomes would perform poorly if they fail to account for this population heterogeneity. We developed criteria for predictive model development using partial surrogate outcomes and demonstrate their importance in model selection and evaluation within the clinical example of serum creatinine, a partial surrogate outcome for acute kidney injury.MethodsData from 4737 patients who underwent cardiac surgery at a major academic center were obtained. Linear and mixture models were fit on maximum 2-day serum creatinine change as a surrogate for estimated glomerular filtration rate at 90 days after surgery (eGFR90), adjusted for known AKI risk factors. The AUC for eGFR90 decline and Spearman’s rho were calculated to compare model discrimination between the linear model and a single component of the mixture model deemed to represent the informative subpopulation. Simulation studies based on the clinical data were conducted to further demonstrate the consistency and limitations of the procedure.ResultsThe mixture model was highly favored over the linear model with BICs of 2131.3 and 5034.3, respectively. When model discrimination was evaluated with respect to the partial surrogate, the linear model displays superior performance (p < 0.001); however, when it was evaluated with respect to the target outcome, the mixture model approach displays superior performance (AUC difference p = 0.002; Spearman’s difference p = 0.020). Simulation studies demonstrate that the nature of the heterogeneity determines the magnitude of any advantage the mixture model.ConclusionsPartial surrogate outcomes add complexity and limitations to risk score modeling, including the potential for the usual metrics of discrimination to be misleading. Partial surrogacy can be potentially uncovered and appropriately accounted for using a mixture model approach. Serum creatinine behaved as a partial surrogate outcome consistent with two patient subpopulations, one representing patients whose injury did not exceed their renal functional reserve and a second population representing patients whose injury did exceed renal functional reserve.
Highlights
Surrogate outcomes are often utilized when disease outcomes are difficult to directly measure
We examine how serum creatinine change for the assessment of acute kidney injury behaves in this manner
The linear model found all the factors to be significantly associated with eGFR90 change except for a history of diabetes and emergency surgery, which were marginally significant (p = 0.085 and p = 0.063, respectively)
Summary
Surrogate outcomes are often utilized when disease outcomes are difficult to directly measure. When a biological threshold effect exists, surrogate outcomes may only represent disease in specific subpopulations. Patient level clinical risk score development and associated decision support applications are vitally important to modern personalized medicine. For many pathologies, this process is straightforward. The disease process of interest is defined and the data about relevant covariates are collected This information is used to develop a statistical model that meets desired performance measures. When the disease process is difficult to directly measure, surrogate measurements are often used which prevent the use of simple risk score modeling methodology. Surrogate outcome criteria for the development of risk scores, remain undefined.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have