Comparing the use of direct observation, standardized patients and exit interviews in low- and middle-income countries: a systematic review of methods of assessing quality of primary care.

Navneet Aujla,Jo Sartori,Anna Wilson,Timothy P Hofer,Frances Griffiths,Yen-Fu Chen,Celia Brown,Yasara Samarakoon,Christian Mallen,Abimbola Ayorinde,Paramjit Gill,Natalia Grolmusová,Richard J Lilford

doi:10.1093/heapol/czaa152

Abstract

Clinical records in primary healthcare settings in low- and middle-income countries (LMIC) are often lacking or of too poor quality to accurately assess what happens during the patient consultation. We examined the most common methods for assessing healthcare workers’ clinical behaviour: direct observation, standardized patients and patient/healthcare worker exit interview. The comparative feasibility, acceptability, reliability, validity and practicalities of using these methods in this setting are unclear. We systematically review and synthesize the evidence to compare and contrast the advantages and disadvantages of each method. We include studies in LMICs where methods have been directly compared and systematic and narrative reviews of each method. We searched several electronic databases and focused on real-life (not educational) primary healthcare encounters. The most recent update to the search for direct comparison studies was November 2019. We updated the search for systematic and narrative reviews on the standardized patient method in March 2020 and expanded it to all methods. Search strategies combined indexed terms and keywords. We searched reference lists of eligible articles and sourced additional references from relevant review articles. Titles and abstracts were independently screened by two reviewers and discrepancies resolved through discussion. Data were iteratively coded according to pre-defined categories and synthesized. We included 13 direct comparison studies and eight systematic and narrative reviews. We found that no method was clearly superior to the others—each has pros and cons and may assess different aspects of quality of care provision by healthcare workers. All methods require careful preparation, though the exact domain of quality assessed and ethics and selection and training of personnel are nuanced and the methods were subject to different biases. The differential strengths suggest that individual methods should be used strategically based on the research question or in combination for comprehensive global assessments of quality.

Highlights

Improving healthcare quality is a major global public health challenge in low- and middle-income countries (LMICs) (United Nations, 2015) and a recent report argues that quality of care has overtaken access to healthcare as the largest problem facing health systems in LMICs (Kruk et al, 2018)
We present a comparative overview of the feasibility, acceptability, validity, reliability, ethics, resources and costs involved in using these methods in the LMIC primary care setting
We carried out two systematic reviews: the first review focuses on primary studies carried out in LMICs that compare one or more of direct observation, standardized patients and exit interviews head to head

Summary

Introduction

Improving healthcare quality is a major global public health challenge in low- and middle-income countries (LMICs) (United Nations, 2015) and a recent report argues that quality of care has overtaken access to healthcare as the largest problem facing health systems in LMICs (Kruk et al, 2018). We concentrate on process, which can be broken down into processes carried out at the system level, such as use of audit and feedback or improving staff morale, and clinical processes impacting directly on patients, such as questions asked to make a diagnosis or prescribe a treatment (Lilford et al, 2010). We refer to the latter as the technical quality of care, corresponding with the definition provided by Donabedian (1988)

Objectives

Methods

Results

Discussion

Conclusion