Model selection for medical diagnosis decision support systems

Abstract

In this paper, we examine the model section decision for a medical diagnostic decision support system (MDSS). Our purpose in doing this is to understand how model selection affects the accuracy of the decision support system. We explore two related research questions: (1) Do ensembles of models, acting as a single decision maker, perform more accurately than single models; and (2) How does model diversity affect the accuracy of the ensembles? Specifically, we compare 23 single models and bootstrap aggregating (i.e., bagging) models for their predictive abilities across five diverse medical data sets. We are able to reach important conclusions about our research objectives. Ensembles are more accurate than single models in their predictive ability. The best ensemble model achieves an error level significantly lower than the error of the best single model for four of the five medical applications analyzed. The magnitude of the error reduction ranges from 6.4% to 17.5%. Also, when designing an ensemble for an MDSS, the decision to diversify the model selection should be guided by the relationship between model instability and generalization error for the population of models under consideration.