The Use of Real World Data for Nice Decision-Making: A Review of Submissions To Three Treatment Effectiveness Evaluation Programmes In 2015 And 2016

Abstract

Randomised controlled trials (RCTs) are traditionally the ‘gold-standard’ data source for National Institute for Care and Health Excellence (NICE) evidence appraisals, although real world data (RWD) is gradually being accepted alongside RCTs or sometimes as a distinct evidence source. Guidance regarding the appropriate use of RWD is limited and varies across programmes. This audit aimed to quantify the extent to which RWD was considered in submissions and reviews for three NICE evaluation programmes in the previous two years. A systematic review of NICE guidance published by the NICE Technology Appraisal (TA), Diagnostic Assessment (DA) and Medical Technologies Evaluation (MTE) programmes in 2015-16 was conducted in May 2017. Key information, including a description of evidence considered, committee comments and recommendations made was extracted and recorded in an electronic database. References to RWD (i.e. data generated from non-comparative, observational studies) were noted. In 2015-16, 91 TA, 9 DA and 9 MTE submissions were appraised by NICE. Of TA evidence submissions, 10% (9/91) included RWD, compared with 67% (6/9) DA and 89% (8/9) MTE submissions. The reviewing committee identified aspects of real-world study methods or reporting requiring improvement for 56% (5/9) TA, 50% (3/6) DA and 44% (4/9) MTE appraisals involving RWD: key areas of focus included sample size, generalisability to wider population, and potential confounders. Guidance included specific recommendations for further RWD collection for 5% TA, 100% DA and 56% MTE submissions. The relatively low proportion of RWD in TA submissions suggests a slower uptake for medicine appraisals compared with diagnostics and other technologies. Nevertheless, the thorough evaluations of real-world evidence by NICE, and trend towards specific recommendations for further RWD, indicate increasing prominence of RWD for NICE decision-making. Detailed guidance and collaboration across NICE programmes is needed to enable robust design and consistent critique of real-world studies.