Combining individual patient data from multiple trials require specific modelling techniques in order to provide accurate predictions

Abstract

We read with interest the article by Pasquier and colleagues [[1]Pasquier M. Hugli O. Pall P. Darocha T. Blancher M. Husby P. et al.Hypothermia outcome prediction after extracorporeal life support for hypothermic cardiac arrest: the HOPE score.Resuscitation. 2018; 126: 58-64Abstract Full Text Full Text PDF PubMed Scopus (58) Google Scholar] on their prediction model for survival in accidental hypothermia treated with extracorporeal circulatory support. This is a rarely encountered population in clinical practice and a predictive model has a noteworthy application. However, we would point out that there are a number of methodological issues with this study that may lead to a distortion in the predictive model estimates. Firstly, the search results have identified 18 studies with a total of 286 patients. The search results are missing a significant number of published papers in the literature. Our recent individual patient data meta-analysis on the same population has identified 84 studies with a total of 658 patients [[2]Saczkowski R. Brown D. Abu-Laban R. Fradet G. Schulze C. Kuzak N. Prediction and risk stratification of survival in accidental hypothermia requiring extracorporeal life support: an individual patient data meta-Analysis.Resuscitation. 2018; 127: 51-57Abstract Full Text Full Text PDF PubMed Scopus (31) Google Scholar]. Secondly, all of the patients appear to have been pooled into one dataset and analyzed with a multivariable logistic regression model. However, this data is multi-level data and each study should be treated as a cluster [[3]Stewart G. Altman D. Askie L. Duley L. Simmonds M. Stewart L. Statistical analysis of individual participant data met-analysis: a comparison of methods and recommendations for practice.PLoS One. 2012; 7: e46042https://doi.org/10.1371/journal.pone.0046042Crossref PubMed Scopus (113) Google Scholar]. There was no accounting for the hierarchical structure of the data in the analysis. The appropriate technique is to analyze the dataset with a logistic regression model that accounts for patient clustering (ie: multi-level model) [[4]Austin P. Merlo J. Intermediate and advanced topics in multilevel logistic regression analysis.Stat Med. 2017; 36: 3257-3277Crossref PubMed Scopus (165) Google Scholar]. By not using a cluster adjustment the model may be miss-specified and the predictor estimates erroneous. Thirdly, there were 5 articles (n = 160) excluded that contained aggregate data and not individual patient data (eTable 1). The authors chose to exclude this data and not use a method to combine aggregate and individual data [[5]Donegan S. Williamson P. D'Alessandro U. Garner P. Smith C. Combining individual patient data and aggregate data in mixed treatment comparison meta‐analysis: individual patient data may be beneficial if only for a subset of trials.Stat Med. 2013; 32: 914-930Crossref PubMed Scopus (52) Google Scholar]. Exclusion of these cases from the analysis and not assessing the effect of adding the aggregate data may have additionally biased the models predictive performance. As such, we believe that the independent predictors of survival may be inaccurate and the resulting risk scoring system should be interpreted with caution. The authors have no conflicts to declare.