Abstract
BackgroundProviders use risk-adjustment systems to help manage healthcare costs. Typically, ordinary least squares (OLS) models on either untransformed or log-transformed cost are used. We examine the predictive ability of several statistical models, demonstrate how model choice depends on the goal for the predictive model, and examine whether building models on samples of the data affects model choice.MethodsOur sample consisted of 525,620 Veterans Health Administration patients with mental health (MH) or substance abuse (SA) diagnoses who incurred costs during fiscal year 1999. We tested two models on a transformation of cost: a Log Normal model and a Square-root Normal model, and three generalized linear models on untransformed cost, defined by distributional assumption and link function: Normal with identity link (OLS); Gamma with log link; and Gamma with square-root link. Risk-adjusters included age, sex, and 12 MH/SA categories. To determine the best model among the entire dataset, predictive ability was evaluated using root mean square error (RMSE), mean absolute prediction error (MAPE), and predictive ratios of predicted to observed cost (PR) among deciles of predicted cost, by comparing point estimates and 95% bias-corrected bootstrap confidence intervals. To study the effect of analyzing a random sample of the population on model choice, we re-computed these statistics using random samples beginning with 5,000 patients and ending with the entire sample.ResultsThe Square-root Normal model had the lowest estimates of the RMSE and MAPE, with bootstrap confidence intervals that were always lower than those for the other models. The Gamma with square-root link was best as measured by the PRs. The choice of best model could vary if smaller samples were used and the Gamma with square-root link model had convergence problems with small samples.ConclusionModels with square-root transformation or link fit the data best. This function (whether used as transformation or as a link) seems to help deal with the high comorbidity of this population by introducing a form of interaction. The Gamma distribution helps with the long tail of the distribution. However, the Normal distribution is suitable if the correct transformation of the outcome is used.
Highlights
IntroductionOrdinary least squares (OLS) models on either untransformed or log-transformed cost are used
Providers use risk-adjustment systems to help manage healthcare costs
The Gamma Log model performed the worst, with a root mean square error (RMSE) twice the value of the RMSE for Square-root Normal model and with confidence intervals that are outside the bounds of all other models' bootstrap confidence intervals (BCIs)
Summary
Ordinary least squares (OLS) models on either untransformed or log-transformed cost are used. While the average number of unique diagnoses (including physical and psychiatric) in the general VA population is 9[3], it is about 14 for patients with MH/SA disorders. Because these patients are most likely to have high resource utilization and consumption, adequate risk-adjustment models are needed to accurately predict MH/SA costs and to better allocate resources among competing facilities and networks within the VA. Two-part models have been suggested as a method for solving the problem where data contain a high percentage of zero costs[4,5]. Transforming the dependent variable introduces retransformation problems related to the original scale[6,7,8]
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.