A meta-analysis but not a systematic review: an evaluation of the Global BMI Mortality Collaboration

Abstract

Meta-analyses of individual participant data (MIPDs) offer many advantages and are considered the highest level of evidence. However, MIPDs can be seriously compromised when they are not solidly founded upon a systematic review. These data-intensive collaborative projects may be led by experts who already have deep knowledge of the literature in the field and of the results of published studies and how these results vary based on different analytical approaches. If investigators tailor the searches, eligibility criteria, and analysis plan of the MIPD, they run the risk of reaching foregone conclusions. We exemplify this potential bias in a MIPD on the association of body mass index with mortality conducted by a collaboration of outstanding and extremely knowledgeable investigators. Contrary to a previous meta-analysis of group data that used a systematic review approach, the MIPD did not seem to use a formal search: it considered 239 studies, of which the senior author was previously aware of at least 238, and it violated its own listed eligibility criteria to include those studies and exclude other studies. It also preferred an analysis plan that was also known to give a specific direction of effects in already published results of most of the included evidence. MIPDs where results of constituent studies are already largely known need safeguards to their validity. These may include careful systematic searches, adherence to the Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data guidelines, and exploration of the robustness of results with different analyses. They should also avoid selective emphasis on foregone conclusions based on previously known results with specific analytical choices.