Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes

Abstract

Inspection systems are used in health care to promote quality improvements, i.e. to achieve changes in organisational structures or processes, healthcare provider behaviour and patient outcomes. These systems are based on the assumption that externally promoted adherence to evidence-based standards (through inspection/assessment) will result in higher quality of health care. However, the benefits of external inspection in terms of organisational, provider and patient level outcomes are not clear. To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes. We searched the following electronic databases for studies: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, Scopus, HMIC, Index to Theses and Intute from their inception dates up to May 2011. There was no language restriction and studies were included regardless of publication status. We searched the reference lists of included studies and contacted authors of relevant papers, accreditation bodies and the International Organization for Standardisation (ISO), regarding any further published or unpublished work. We included randomised controlled trials (RCTs), controlled clinical trials (CCTs), interrupted time-series (ITSs) and controlled before and after studies (CBAs) evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes in hospitals, primary healthcare organisations and other community-based healthcare organisations. Two review authors independently applied eligibility criteria, extracted data and assessed the risk of bias of each included study. Since meta-analysis was not possible, we produced a narrative results summary. We identified one cluster-RCT involving 20 South African public hospitals (Salmon 2003) and one ITS involving all acute trusts in England (OPM 2009) for inclusion in this review.Salmon and colleagues (Salmon 2003) showed mixed effects of a hospital accreditation system on the compliance with COHSASA (the Council for Health Services Accreditation for South Africa) accreditation standards and eight indicators of hospital quality. Significantly improved total mean compliance score with COHSASA accreditation standards was found for 21/28 service elements: mean intervention effect (95% confidence interval (CI)) was 30% (23% to 57%) (P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). A sub-analysis of 424 a priori identified critical criteria (19 service elements) showed significantly improved compliance with the critical standards (P < 0.001). The score increased from 41% (21% to 46%) to 75% (55% to 96%) in intervention hospitals, but was unchanged in control hospitals (37%). Only one of the nine intervention hospitals gained full accreditation status at the end of the study period, with two others reached pre-accreditation status.The median intervention effect (range) for the indicators of hospital quality of care was 2.4 (-1.9 to +11.8) and only one of the eight indicators: 'nurses perception of clinical quality, participation and teamwork' was significantly improved (mean intervention effect 5.7, P = 0.03).Re-analysis of the MRSA (methicillin-resistant Staphylococcus aureus) data showed statistically non-significant effects of the Healthcare Commissions Infection Inspection programme. We only identified two studies for inclusion in this review, which highlights the paucity of high-quality controlled evaluations of the effectiveness of external inspection systems. No firm conclusions could therefore be drawn about the effectiveness of external inspection on compliance with standards.