Linking clinical quality indicators to research evidence - a case study in asthma management for children

Miew Keen Choong,William B Runciman,Peter Hibbert,Guy Tsafnat,Enrico Coiera

doi:10.1186/s12913-017-2324-y

Abstract

BackgroundClinical quality indicators are used to monitor the performance of healthcare services and should wherever possible be based on research evidence. Little is known however about the extent to which indicators in common use are based on research. The objective of this study is to measure the extent to which clinical quality indicators used in asthma management in children with outcome measurements can be linked to results in randomised controlled clinical trial (RCT) reports. This work is part of a broader research program to trial methods that improve the efficiency and accuracy of indicator development.MethodsNational-level indicators for asthma management in children were extracted from the National Quality Measures Clearinghouse database and the National Institute for Health and Care Excellence quality standards by two independent appraisers. Outcome measures were extracted from all published English language RCT reports for asthma management in children below the age of 12 published between 2005 and 2014. The two sets were then linked by manually mapping both to a common set of Unified Medical Language System (UMLS) concepts.ResultsThe analysis identified 39 indicators and 562 full text RCTs dealing with asthma management in children. About 95% (37/39) of the indicators could be linked to RCT outcome measures.ConclusionsIt is possible to identify relevant RCT reports for the majority of indicators used to assess the quality of asthma management in childhood. The methods reported here could be automated to more generally support assessment of candidate indicators against the research evidence.

Highlights

Clinical quality indicators are used to monitor the performance of healthcare services and should wherever possible be based on research evidence
As a case study to support the development of methods to improve the efficiency and accuracy of indicator development, we examine the relationship between national level clinical quality indicators for asthma management in pre-adolescent children in the UK and the USA and recent randomised controlled clinical trial (RCT) reports
For this study we used a four-stage protocol for extracting indicators and linking them to outcome measures in the research literature (Fig. 1) [21]: (1)Identification and extraction of indicators from the USA and the UK; (2)Identification and extraction of outcome measures from RCTs; (3)Mapping indicators and RCT outcome measures to a standard clinical concept set within the Unified Medical Language System (UMLS), using the MetaMap tool [22]; (4)Evaluating whether the mapped indicators could be associated with RCT outcome measures

Summary

Introduction

Clinical quality indicators are used to monitor the performance of healthcare services and should wherever possible be based on research evidence. Little is known about the extent to which indicators in common use are based on research. The drive to improve the quality and safety of healthcare has resulted in a proliferation of clinical quality indicators. A good indicator should say “as much about a system as possible in as few points as possible” [3]. It should be important, relevant, valid, reliable, meaningful and understandable [3, 4].

Objectives

Methods

Results

Discussion

Conclusion