PHP253 - The Challenges of Carrying out an Indirect Comparison with Single-Arm Studies

Abstract

To assess guidance for using single-arm studies in indirect comparisons (ICs) to support health technology assessment submissions. ICs enable comparison of different treatments using data from separate studies. The published methodology for conducting ICs outline the requirement for multiple arm studies to form connections between direct and indirect evidence. This is not possible when trials have single arms, a particular issue for treatments for new indications or for rare diseases. Guidelines were searched for advice on the use of single-arm trial data in ICs: National Institute of Health and Care Excellence [NICE], Cochrane; Centre for Reviews and Dissemination; and Preferred Reporting Items for Systematic Reviews and Meta-Analyses. In addition the NICE website was searched for new drug submissions including single-arm studies in ICs. There is minimal current guidance for dealing with single-arm studies in ICs. NICE recommends that such analyses should be treated as observational and appropriate steps taken to adjust for possible bias. There are very few submissions where single-arm trial evidence has been used in an IC. Those that attempted to compare outcomes from single-arm studies were mostly in oncology and in many cases the reviewing committee had requested this additional information. Cost-effectiveness calculations using these data were acknowledged as being highly unreliable, even after attempts had been made to adjust for possible between-study bias. A number of methods could be used to compare results across multiple single-arm studies. A Bayesian hierarchical model that includes random effects allowing for heterogeneity between studies is a good choice. However, there is a need for clear guidance from organisations such as NICE and Cochrane, who provide recommendations on carrying out systematic reviews and indirect comparisons, on this and other approaches to synthesising information from single-arm studies.