Validation of logistic regression models used in the assessment of prognosis and the need for surgery in equine colic patients

Abstract

An independent validation study was performed to assess the predictive performance of two multivariable logistic regression models which had been developed to evaluate both prognosis and the need for surgery in equine colic patients. The validation study prospectively collected 730 equine colic cases at 16 different locations over a 15-month period. Bayes' theorem was used to generate post-test (posterior) probabilities for each outcome. Likelihood ratios (LR) for death and the need for surgery were calculated for each horse using the two logistic regression equations. The proportion of surgical patients and the case fatality rate of colic cases seen at each institution were used to estimate the pre-test odds of the need for surgery and the pre-test odds of death, respectively. The post-test odds then were generated by calculating the product of the pre-test odds and the LR (test-odds). The post-test odds than were simply converted to the more intuitive post-test or posterior probabilities. Hosmer Lemeshow goodness-of-fit chi square statistics (GOFCS) were calculated using equal-sized deciles of risk for both models. The prognosis model fit the validation data well ( χ 2 = 11.2, df = 10, 0.3 < P < 0.4), while the need-for-surgery model fit the validation data poorly ( χ 2 = 39.0, df = 10, P = < 0.001). An additional validation procedure was used in which the post-test probabilities were ranked and then grouped into ten increments based on the following cut-points of post-test probability: 0−<0.1; 0.1 −<0.2−<0.3;…; 0.9–1.0. The observed proportion of deaths and the observed proportion of surgical cases were compared with the expected proportions (i.e. predicted by the logistic regression models) for each increment. The prognosis model again fit the data well, while the surgery model tended to overestimate the need for surgery within each increment. Procedures for the practical implementation of this methodology to clinical environments are being pursued.