Meta-analysis of clinical prediction models

Abstract

The past decades there has been a clear shift from implicit to explicit diagnosis and prognosis. This includes appreciation of clinical -diagnostic and prognostic- prediction models, which is likely to increase with the introduction of fully computerized patient records. Prediction models aim to provide a probability of disease/outcome presence (diagnosis) or occurrence (prognosis) in an individual. Unfortunately, there are many examples of prediction models that show optimistic accuracy in the data from which they were developed. They show substantially lower accuracy when validated in new populations, compromising patient management and outcome. It is widely agreed that each prediction model should be validated before application in practice. In case of poor accuracy in the validation data, investigators often proceed to develop their ‘own’ prediction model. This urge to develop a new model from each dataset at hand is an unfortunate habit; it makes prediction research particularistic and prior knowledge is not optimally used. Moreover, validation studies are often smaller than development studies, such that the accuracy of the new model (from the validation set) in future patients can actually be worse than applying the original model. Instead, meta-analysis-like methods may be considered to synthesize available evidence from previously published models and derive a more up-to-date and better generalizable model. We propose that one should start from a (well developed) multivariable model - obtained from either a single study or from a meta-analytical approach of several studies - and update this model with the validation data, to enhance its accuracy in future patients.