55 Assessment of methodological quality of animal and human studies of a blockbuster immunotherapy drug ipilimumab: a systematic review

Abstract

Objectives Up to 85% of biomedical research investment is lost and majority of the waste can be attributed to inadequate study methodology. The lack reproducibility of studies especially animal studies is worrisome and may add to research waste. Ipilimumab is a human monoclonal antibody that targets cytotoxic T-lymphocyte antigen-4 (CTLA-4) to promote T-cell mediated anti-tumour responses. It was approved for the treatment of metastatic melanoma by the Food and Drug Administration (FDA) and European Medicines Agency (EMA) in 2011. Many studies have evaluated the safety and efficacy of Ipilimumab in the treatment of melanoma and other cancers. Optimally, the implicit quality of these studies would ensure patient safety, enhance the soundness of clinical knowledge and practice, and reduce research waste. In this systematic review, we evaluated the methodological quality of animal and human trials investigating efficacy or toxicity of a successful immunotherapeutic drug: Ipilimumab. Method All randomised clinical trials studies investigating Ipilimumab/anti-CTLA4 in human or animal models of any cancer type with primary outcomes of efficacy or safety were included. A comprehensive literature search first via PubMed, EMBASE, and the CENTRAL databases and then using snow ball strategy was conducted. No time or language limits were used. The methodological quality of human studies was investigated using the Cochrane Risk of Bias assessment and methodological quality of animal studies was assessed employing the ARRIVE guidelines. Each domain in the ARRIVE guideline was also categorised as high versus low versus unclear risk of bias. Results 29 studies (15 animal and 14 human) were included. Among animal studies categories that exhibited high risk of bias due to lack of reporting were: baseline data (15/15), description of methods of allocating animals to experimental groups and order in which the animals in the different experimental groups were treated (12/15), and description of adverse events and resulting modifications to protocol (11/15). Categories with unclear risk of bias were: experimental procedures (15/15), experimental animals (details of the animals used, including species, strain, sex, developmental stage and weight), study design (13/15) each, outcomes and estimation; study title (11/15) each, abstract, background (9/15) each, and objectives (8/15). In human studies categories with high risk of bias were: incomplete outcome data (4/14) and other sources of biases (3/14). Categories with unclear risk of bias were: random sequence generation (10/14), allocation concealment, blinding of participants and personnel, and blinding of outcome assessment (9/14) each. Conclusions Our study clearly highlights low to poor methodological quality of both animal and human studies used in investigating efficacy and safety of blockbuster drug Ipilimumab, including those studies used in FDA and EMA approval processes. The observed low/poor quality may lead to research waste and can be avoided in part by modest and reasonable methodological adjustments during study design phase. Clear, comprehensive reporting especially of animal studies could decrease waste of research and foster evidence-based clinical practice including replication of experiments leading to robust evidence base.