Audit and feedback: effects on professional practice and healthcare outcomes.

Cochrane Central Register of Controlled Trials (CENTRAL), the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register, Medline, Ovid; Embase, CINAHL, EBSCO, Science Citation Index and Social Sciences Citation Index, ISIWeb of Science Randomised trials of audit and feedback that reported objectively measured health professional practice or patient outcomes. Data were abstracted by two independent review authors using a data extraction form. The following factors were examined as possible explanations for the variation in the effectiveness of interventions across comparisons; format of feedback, source of feedback, frequency of feedback, instructions for improvement, direction of change required, baseline performance, profession of recipient and risk of bias within the trial itself. Risk of bias was assessed using EPOC criteria. The degree of confidence in the estimate of effect across studies was assessed using the GRADE approach. All outcomes were expressed as compliance with desired practice and both professional and patient outcomes were assessed, though separately. One hundred and forty studies were included. Using 70 of these 108 comparisons were made. After excluding studies at high risk of bias there were 82 comparisons from 49 studies with dichotomous outcomes, and the weighted median adjusted RD was 4.3% (interquartile range (IQR) 0.5% to 16%). For studies with continuous outcomes the weighted median adjusted percentage change relative to control was 1.3% (IQR = 1.3%to 28.9%). For patient outcomes the weighted median RD was 0.4% (IQR -1.3% to 1.6%) in studies reporting dichotomous outcomes, and the weighted median percentage change was 17% (IQR 1.5% to 17%) for studies with continuous outcomes. Multivariable meta-regression indicated that feedback may be more effective when baseline performance is low, the source is a supervisor or colleague, it is provided more than once, it is delivered in both verbal and written formats and when it includes both explicit targets and an action plan. Audit and feedback generally lead to small but potentially important improvements in professional practice.

Audit and feedback continues to be widely used as a strategy to improve professional practice. It appears logical that healthcare professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines. Yet, audit and feedback has not consistently been found to be effective. To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes. We searched the Cochrane Effective Practice and Organisation of Care Group's register and pending file up to January 2004. Randomised trials of audit and feedback (defined as any summary of clinical performance over a specified period of time) that reported objectively measured professional practice in a healthcare setting or healthcare outcomes. Two reviewers independently extracted data and assessed study quality. Quantitative (meta-regression), visual and qualitative analyses were undertaken. For each comparison we calculated the risk difference (RD) and risk ratio (RR), adjusted for baseline compliance when possible, for dichotomous outcomes and the percentage and the percent change relative to the control group average after the intervention, adjusted for baseline performance when possible, for continuous outcomes. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: the type of intervention (audit and feedback alone, audit and feedback with educational meetings, or multifaceted interventions that included audit and feedback), the intensity of the audit and feedback, the complexity of the targeted behaviour, the seriousness of the outcome, baseline compliance and study quality. Thirty new studies were added to this update, and a total of 118 studies are included. In the primary analysis 88 comparisons from 72 studies were included that compared any intervention in which audit and feedback is a component compared to no intervention. For dichotomous outcomes the adjusted risk difference of compliance with desired practice varied from - 0.16 (a 16 % absolute decrease in compliance) to 0.70 (a 70% increase in compliance) (median = 0.05, inter-quartile range = 0.03 to 0.11) and the adjusted risk ratio varied from 0.71 to 18.3 (median = 1.08, inter-quartile range = 0.99 to 1.30). For continuous outcomes the adjusted percent change relative to control varied from -0.10 (a 10 % absolute decrease in compliance) to 0.68 (a 68% increase in compliance) (median = 0.16, inter-quartile range = 0.05 to 0.37). Low baseline compliance with recommended practice and higher intensity of audit and feedback were associated with larger adjusted risk ratios (greater effectiveness) across studies. Audit and feedback can be effective in improving professional practice. When it is effective, the effects are generally small to moderate. The relative effectiveness of audit and feedback is likely to be greater when baseline adherence to recommended practice is low and when feedback is delivered more intensively.

Educational meetings are used widely by health personnel to provide continuing medical education and to promote implementation of innovations or translate new knowledge to change practice within healthcare systems. Previous reviews have concluded that educational meetings can result in small changes in behaviour, but that effects vary considerably. Investigations into which characteristics of educational meetings might lead to greater impact have yielded varying results, and factors that might explain heterogeneity in effects remain unclear. This is the second update of this Cochrane Review.

Review question/objective The objective of this systematic review is to synthesize the best available evidence for the effectiveness of audit and feedback as a quality improvement strategy. More specifically, the objectives are to identify: 1. What characteristics contribute to an effective audit and feedback cycle in aged care. 2. Whether audit and feedback is an effective quality improvement strategy for improving health outcomes for older people in residential aged care facilities. 3. What the most effective feedback mechanisms as part of an audit and feedback cycle in implementing practice change are. Background Systematic reviews of the effectiveness of audit and feedback as a quality improvement strategy to improve practice and health care outcomes have been reported since 2000 with varying effects being identified.1 Traditionally, these reviews have examined medical practices and the acute health care sector. The most recent systematic review - conducted in 2012 - on audit and feedback outlined that this process has been widely used as a quality improvement tool and concluded that the implication for practice “generally leads to small but potentially important improvements in professional practice”.2(p.2) It also concluded that “the effectiveness of feedback seems to depend on baseline performance and how feedback is provided”.2(p.2) This review only included randomized controlled trials, the participants were healthcare professionals including physicians, pharmacists and nurses responsible for patient care and it included both inpatient and outpatient settings.2 Audit and feedback and clinical audit appear to be used interchangeably in the literature. In the United Kingdom a clinical audit was seen as one approach to improving the quality of patient care and was introduce in the 1990s. One article defined clinical audit as: ‘the systematic, critical analysis of the quality of medical care, including the procedures used for diagnosis and treatment, the use of resources, and the resulting outcome and quality of life for the patient’ or “audit is the process of reviewing the delivery of care to identify deficiencies so that they may be remedied”.1(p.24) PubMed defines clinical audit as ‘a detailed review and evaluation of selected clinical records by qualified professional personnel to improve the quality of patient care and outcome’. Interestingly there was no definition on PubMed for audit and feedback. Audit and feedback was defined in a systematic review in 2006 as “the provision of any summary of clinical performance over a specified period of time”.3 However, Pearson et al. 2007 provide a clearer description of clinical audit describing it as a tool that addresses all components of clinical effectiveness where health professionals examine their care practices against clinical guidelines or best practice statements. Pearson et al also explain that clinical effectiveness includes “best-practice activities such as systematic reviews, clinical guidelines and clinical audit”.4(p.134) The authors describe the clinical audit and feedback process as a method of promoting evidence utilization and how it can be utilized an “internal mechanism for health professionals to use for quality improvement”.4(p.138) However, an initial search of various databases indicated that the term clinical audit is not commonly used in studies involving residential aged care facilities and more often the term audit and feedback is used. Therefore this review will include both audit and feedback, and clinical audit, in its search strategy focusing on implementation of evidence-based practice. A literature review conducted in 2009, identified eight factors that might influence the implementation (utilization) of evidence-based practice in aged care in Australia.5 These included: “a receptive context for change; having a model of change to guide implementation; adequate resources; staff with the necessary skills; stakeholder engagement, participation and commitment; the nature of the change in practice; systems in place to support the use of evidence; and demonstrable benefits of the change”.5 Whilst this review did not specifically look at audit and feedback the underlying assumption was that ‘evidence-based practice is about taking something new (the evidence) from one domain (research) and implementing it in another domain (practice), which describes a process used in clinical audit.5 Ives et al also stated: “in an audit and feedback process, an individual's professional practice or performance is measured and then compared to professional standards or targets”.2(p.2) There is evidence to confirm that audit and feedback on its own or as part of a multifaceted change process can have some effect on health outcomes and practice change.1 Implementation and utilization of evidence-based practice is a challenge for researchers, educators, policy makers and health practitioners. A review that can provide evidence that the audit and feedback cycle, when used as a quality improvement strategy, can impact on clinical outcomes for a high risk population with co-morbidities, is warranted. This is supported by Masso's literature review that outlines the Encouraging Best Practice in Residential Aged Care (EBPRAC) program, funded by the Australian Government, which was introduced in 2007 to implement evidence-based practice in residential aged care facilities.5 The program involved 13 projects working with facilities in 108 locations across six states.5 The impact of this project is outlined in the Encouraging Best Practice in Residential Aged Care Program: Final Evaluation Report (2011) which states “collectively, the projects had a positive impact on residents, staff and facilities, with considerable variation between individual projects, and within each project”.6(p.97) Changes to the care received by residents were diverse. Many of the changes built on work that had been done previously in participating facilities and were relatively small scale and incremental in nature. In part this reflects the focus of the program and the available evidence. It is also indicative of the capacity of the sector to change; however this is dependent on the availability of resources, including the knowledge and skills of staff, the nature of daily work and the influence that a wide range of factors that are largely outside the control of those trying to bring about change can have e.g. turnover of facility managers which had a significant impact on some facilities and some projects.6 This report and the literature review by Masso et al. both confirm that aged care as a profession is willing to change depending on a number of variables as outlined in both papers.5,6 A total of $12,918,238 in Commonwealth funding was allocated to round one and two of this project.6 In 2010, EBPRAC was expanded to include community aged care that consequently resulted in the name being change to: ‘Encouraging Better Practice in Aged Care’ (EBPAC) program. In the 2011-2012 Australian federal budget, the EBPAC initiative was incorporated into the Aged Care Services Improvement and Healthy Ageing Grants Fund. This confirms that implementing better practice in aged care is still on the political agenda and funding is being made available. The Aged Care Standards and Accreditation Agency (previously the auditing body for aged care in Australia) has facilitated the “Better Practice” in aged care award annually. The criteria for an award were as follows: Is the program based on the findings of research/feedback? Has the program's effectiveness been measured and monitored? Is the program focused on achieving better outcomes for the residents? Does the program demonstrate evolving and improving, finding the best way of doing things? How does it link with the residential aged care home's continuous improvement system? This demonstrates that implementing better practice is being supported by funding and auditing bodies nationally in Australia. A systematic review of audit and feedback as an evidence-based quality improvement strategy is essential to provide the aged care industry (including policy makers, educators, medical professionals, allied health and nursing professionals, care staff, residents, families and consumers) with a review of the effectiveness of this process in improving health outcomes and the implementation of evidence-based practice. A study using a cross-sectional post feedback survey, conducted in Canada and published in 2013, examined unregulated care provider's perceptions of audit and feedback reports.7 This report concluded that “unregulated care providers can understand and feel positively about using audit with feedback reports to make changes to resident care”.7(p.1) This report recommended “further research should explore ways to promote fuller engagement of unregulated care providers in decision-making to improve quality of care in long-term care settings”.7(p.1) A search of the Cochrane Library identified 36 systematic reviews on audit and feedback and confirmed that only one of these related to aged care and had reviewed the prevention of the transmission of meticillin-resistant Staphylococcus aureus (MRSA) in nursing homes for older people.8 This search also identified 63 systematic reviews on clinical audit and there were no systematic reviews relevant to residential aged care. A search of the Joanna Briggs Institute Database of Systematic Reviews and Implementation Studies also revealed no systematic reviews or protocols had been conducted on this topic. The most recent systematic review in the Cochrane Library by Ives et al confirms that audit and feedback is effective, however it will “continue to be unreliable approach to quality improvement until we learn how and when it works best”.2(p.5) The lack of literature on audit and feedback and/or clinical audit in aged care suggests that the healthcare sector needs more information on how useful this tool could be as a quality improvement strategy to improve outcomes for residents in their care. Notwithstanding this, there are studies that have used audit and feedback as a change process in aged care some of which are randomized controlled trials. A review of these studies may provide the necessary evidence to increase the uptake of audit and feedback as a quality improvement strategy that may support the implementation and utilization of evidence-based practice. This is a quantitative systematic review as it seeks to establish the effectiveness of audit and feedback as a quality improvement strategy. While it is acknowledged that qualitative data may inform this review, it is not within the proposed scope of this review. Whilst this protocol has predominately reflected on the Australian aged care sector, to obtain enough studies to be included in this review, evidence will be sought globally. It is hope that the lessons learnt internationally will be transferable to the local context as the review is focusing on evidence-based practice and not local legislative requirements.

Educational meetings are widely used for continuing medical education. Previous reviews found that interactive workshops resulted in moderately large improvements in professional practice, whereas didactic sessions did not. To assess the effects of educational meetings on professional practice and healthcare outcomes. We updated previous searches by searching the Cochrane Effective Practice and Organisation of Care Group Trials Register and pending file, from 1999 to March 2006. Randomised controlled trials of educational meetings that reported an objective measure of professional practice or healthcare outcomes. Two authors independently extracted data and assessed study quality. Studies with a low or moderate risk of bias and that reported baseline data were included in the primary analysis. They were weighted according to the number of health professionals participating. For each comparison, we calculated the risk difference (RD) for dichotomous outcomes, adjusted for baseline compliance; and for continuous outcomes the percentage change relative to the control group average after the intervention, adjusted for baseline performance. Professional and patient outcomes were analysed separately. We considered 10 factors to explain heterogeneity of effect estimates using weighted meta-regression supplemented by visual analysis of bubble and box plots. In updating the review, 49 new studies were identified for inclusion. A total of 81 trials involving more than 11,000 health professionals are now included in the review. Based on 30 trials (36 comparisons), the median adjusted RD in compliance with desired practice was 6% (interquartile range 1.8 to 15.9) when any intervention in which educational meetings were a component was compared to no intervention. Educational meetings alone had similar effects (median adjusted RD 6%, interquartile range 2.9 to 15.3; based on 21 comparisons in 19 trials). For continuous outcomes the median adjusted percentage change relative to control was 10% (interquartile range 8 to 32%; 5 trials). For patient outcomes the median adjusted RD in achievement of treatment goals was 3.0 (interquartile range 0.1 to 4.0; 5 trials). Based on univariate meta-regression analyses of the 36 comparisons with dichotomous outcomes for professional practice, higher attendance at the educational meetings was associated with larger adjusted RDs (P < 0.01); mixed interactive and didactic education meetings (median adjusted RD 13.6) were more effective than either didactic meetings (RD 6.9) or interactive meetings (RD 3.0). Educational meetings did not appear to be effective for complex behaviours (adjusted RD -0.3) compared to less complex behaviours; they appeared to be less effective for less serious outcomes (RD 2.9) than for more serious outcomes. Educational meetings alone or combined with other interventions, can improve professional practice and healthcare outcomes for the patients. The effect is most likely to be small and similar to other types of continuing medical education, such as audit and feedback, and educational outreach visits. Strategies to increase attendance at educational meetings, using mixed interactive and didactic formats, and focusing on outcomes that are likely to be perceived as serious may increase the effectiveness of educational meetings. Educational meetings alone are not likely to be effective for changing complex behaviours.

Background: Many people advocate audit and feedback as a strategy for improving professional practice. The main results of an update of a Cochrane review on the effects of audit and...

Educational outreach visits (EOVs) have been identified as an intervention that may improve the practice of healthcare professionals. This type of face-to-face visit has been referred to as university-based educational detailing, academic detailing, and educational visiting. To assess the effects of EOVs on health professional practice or patient outcomes. For this update, we searched the Cochrane EPOC register to March 2007. In the original review, we searched multiple bibliographic databases including MEDLINE and CINAHL. Randomised trials of EOVs that reported an objective measure of professional performance or healthcare outcomes. An EOV was defined as a personal visit by a trained person to healthcare professionals in their own settings. Two reviewers independently extracted data and assessed study quality. We used bubble plots and box plots to visually inspect the data. We conducted both quantitative and qualitative analyses. We used meta-regression to examine potential sources of heterogeneity determined a priori. We hypothesised eight factors to explain variation across effect estimates. In our primary visual and statistical analyses, we included only studies with dichotomous outcomes, with baseline data and with low or moderate risk of bias, in which the intervention included an EOV and was compared to no intervention. We included 69 studies involving more than 15,000 health professionals. Twenty-eight studies (34 comparisons) contributed to the calculation of the median and interquartile range for the main comparison. The median adjusted risk difference (RD) in compliance with desired practice was 5.6% (interquartile range 3.0% to 9.0%). The adjusted RDs were highly consistent for prescribing (median 4.8%, interquartile range 3.0% to 6.5% for 17 comparisons), but varied for other types of professional performance (median 6.0%, interquartile range 3.6% to 16.0% for 17 comparisons). Meta-regression was limited by the large number of potential explanatory factors (eight) with only 31 comparisons, and did not provide any compelling explanations for the observed variation in adjusted RDs. There were 18 comparisons with continuous outcomes, with a median adjusted relative improvement of 21% (interquartile range 11% to 41%). There were eight trials (12 comparisons) in which the intervention included an EOV and was compared to another type of intervention, usually audit and feedback. Interventions that included EOVs appeared to be slightly superior to audit and feedback. Only six studies evaluated different types of visits in head-to-head comparisons. When individual visits were compared to group visits (three trials), the results were mixed. EOVs alone or when combined with other interventions have effects on prescribing that are relatively consistent and small, but potentially important. Their effects on other types of professional performance vary from small to modest improvements, and it is not possible from this review to explain that variation.

Background Printed educational materials are widely used dissemination strategies to improve the quality of healthcare professionals' practice and patient health outcomes. Traditionally they are presented in paper formats such as monographs, publication in peer‐reviewed journals and clinical guidelines. This is the fourth update of the review. Objectives To assess the effect of printed educational materials (PEMs) on the practice of healthcare professionals and patient health outcomes. To explore the influence of some of the characteristics of the printed educational materials (e.g. source, content, format) on their effect on healthcare professionals' practice and patient health outcomes. Search methods We searched MEDLINE, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), HealthStar, CINAHL, ERIC, CAB Abstracts, Global Health, and EPOC Register from their inception to 6 February 2019. We checked the reference lists of all included studies and relevant systematic reviews. Selection criteria We included randomised trials (RTs), controlled before‐after studies (CBAs) and interrupted time series studies (ITSs) that evaluated the impact of PEMs on healthcare professionals' practice or patient health outcomes. We included three types of comparisons: (1) PEM versus no intervention, (2) PEM versus single intervention, (3) multifaceted intervention where PEM is included versus multifaceted intervention without PEM. Any objective measure of professional practice (e.g. prescriptions for a particular drug), or patient health outcomes (e.g. blood pressure) were included. Data collection and analysis Two reviewers undertook data extraction independently. Disagreements were resolved by discussion. For analyses, we grouped the included studies according to study design, type of outcome and type of comparison. For controlled trials, we reported the median effect size for each outcome within each study, the median effect size across outcomes for each study and the median of these effect sizes across studies. Where data were available, we re‐analysed the ITS studies by converting all data to a monthly basis and estimating the effect size from the change in the slope of the regression line between before and after implementation of the PEM. We reported median changes in slope for each outcome, for each study, and then across studies. We standardised all changes in slopes by their standard error, allowing comparisons and combination of different outcomes. We categorised each PEM according to potential effects modifiers related to the source of the PEMs, the channel used for their delivery, their content, and their format. We assessed the risks of bias of all the included studies. Main results We included 84 studies: 32 RTs, two CBAs and 50 ITS studies. Of the 32 RTs, 19 were cluster RTs that used various units of randomisation, such as practices, health centres, towns, or areas. The majority of the included studies (82/84) compared the effectiveness of PEMs to no intervention. Based on the RTs that provided moderate‐certainty evidence, we found that PEMs distributed to healthcare professionals probably improve their practice, as measured with dichotomous variables, compared to no intervention (median absolute risk difference (ARD): 0.04; interquartile range (IQR): 0.01 to 0.09; 3,963 healthcare professionals randomised within 3073 units). We could not confirm this finding using the evidence gathered from continuous variables (standardised mean difference (SMD): 0.11; IQR: ‐0.16 to 0.52; 1631 healthcare professionals randomised within 1373 units ), from the ITS studies (standardised median change in slope = 0.69; 35 studies), or from the CBA study because the certainty of this evidence was very low. We also found, based on RTs that provided moderate‐certainty evidence, that PEMs distributed to healthcare professionals probably make little or no difference to patient health as measured using dichotomous variables, compared to no intervention (ARD: 0.02; IQR: ‐0.005 to 0.09; 935,015 patients randomised within 959 units). The evidence gathered from continuous variables (SMD: 0.05; IQR: ‐0.12 to 0.09; 6,737 patients randomised within 594 units) or from ITS study results (standardised median change in slope = 1.12; 8 studies) do not strengthen these findings because the certainty of this evidence was very low. Two studies (a randomised trial and a CBA) compared a paper‐based version to a computerised version of the same PEM. From the RT that provided evidence of low certainty, we found that PEM in computerised versions may make little or no difference to professionals' practice compared to PEM in printed versions (ARD: ‐0.02; IQR: ‐0.03 to 0.00; 139 healthcare professionals randomised individually). This finding was not strengthened by the CBA study that provided very low certainty evidence (SMD: 0.44; 32 healthcare professionals). The data gathered did not allow us to conclude which PEM characteristics influenced their effectiveness. The methodological quality of the included studies was variable. Half of the included RTs were at risk of selection bias. Most of the ITS studies were conducted retrospectively, without prespecifying the expected effect of the intervention, or acknowledging the presence of a secular trend. Authors' conclusions The results of this review suggest that, when used alone and compared to no intervention, PEMs may slightly improve healthcare professionals' practice outcomes and patient health outcomes. The effectiveness of PEMs compared to other interventions, or of PEMs as part of a multifaceted intervention, is uncertain.

Audit and feedback has been identified as having the potential to change the practice of health care professionals. To assess the effects of audit and feedback compared with other interventions in changing health professional practice and to assess whether the effectiveness of audit and feedback can be improved by modifying how it is done. We searched MEDLINE up to June 1997, the Research and Development Resource Base in Continuing Medical Education, and reference lists of related systematic reviews and articles. Randomised trials of audit and feedback (defined as any summary of clinical performance of health care over a specified period of time) compared with other interventions. The participants were health care providers responsible for patient care. Two reviewers independently extracted data and assessed study quality. Twelve studies were included involving more than 2194 physicians. Seven trials with direct comparisons were included. The targeted behaviours were the management of low haemoglobin, the delivery of preventive care services (two studies), the management of high cholesterol, the performance of cervical smears, and the ordering of diagnostic tests (two studies). From the results of four trials, there is little evidence of a measurable effect of adding a complementary intervention such as a local consensus process to audit and feedback compared to audit and feedback alone. Two of three trials that compared audit and feedback to reminders reported that reminders were more effective in improving the delivery of some preventive services. It is not possible to recommend a complementary intervention to enhance the effectiveness of audit and feedback. Reminders might be more effective than audit and feedback to improve the delivery of some preventive services but the results are not striking. Few trials have investigated the effect of varying different characteristics of the audit and feedback process. Consideration should be given to testing the effects of modifying important characteristics such as the content, source, timing, recipient and format.

Printed educational materials (PEMs) are widely used passive dissemination strategies to improve knowledge, awareness, attitudes, skills, professional practice and patient outcomes. Traditionally they are presented in paper formats such as monographs, publication in peer-reviewed journals and clinical guidelines and appear to be the most frequently adopted method for disseminating information. To determine the effectiveness of PEMs in improving process outcomes (including the behaviour of healthcare professionals) and patient outcomes. To explore whether the effect of characteristics of PEMs (e.g., source, content, format, mode of delivery, timing/frequency, complexity of targeted behaviour change) can influence process outcomes (including the behaviour of healthcare professionals and patient outcomes). The following electronic databases were searched up to July 2006: (a) The EPOC Group Specialised Register (including the database of studies awaiting assessment (see 'Specialised Register'under 'Group Details'); (b) The Cochrane Central Register of Controlled Trials (CENTRAL) and the Database of Abstracts of Reviews of Effectiveness; (c) MEDLINE, EMBASE, CINAHL and CAB Health. An updated search of MEDLINE was done in March 2007. We included randomised controlled trials (RCTs) , controlled clinical trials (CCT), controlled before and after studies (CBAs) and interrupted time series analyses (ITS) that evaluated the impact of printed educational materials on healthcare professionals' practice and/or patient outcomes. There was no language restriction. Any objective measure of professional performance (sch as number of tests ordered, prescriptions for a particular drug), or patient health outcomes (e.g., blood pressure, number of caesarean sections) were included. Four reviewers undertook data abstraction independently using a modified version of the EPOC data collection checklist. Any disagreement was resolved by discussion among the reviewers and arbitrators. Statistical analysis was based upon consideration of dichotomous process outcomes, continuous process outcomes, patient outcome dichotomous measures and patient outcome continuous measures. We presented the results for all comparisons using a standard method of presentation where possible. We reported separately for each study the median effect size for each type of outcome, and the median of these effect sizes across studies. Twenty-three studies were included for this review. Evidence from this review showed that PEMs appear to have small beneficial effects on professional practice. RCTs comparing PEMs to no intervention observed an absolute risk difference median: +4.3% on categorical process outcomes (e.g., x-ray requests, prescribing and smoking cessation activities) (range -8.0% to +9.6%, 6 studies), and a relative risk difference +13.6% on continuous process outcomes (e.g., medication change, x-rays requests per practice) (range -5.0% to +26.6%, 4 studies). These findings are similar to those reported for the ITS studies, although significantly larger effect sizes were observed (relative risk difference range from 0.07% to 31%). In contrast, the median effect size was -4.3% for patient outcome categorical measures (e.g., screening, return to work, quit smoking) (range -0.4% to -4.6%, 3 studies)). Two studies reported deteriorations in continuous patient outcome data (e.g., depression score, smoking cessation attempts) of -10.0% and -20.5%. One study comparing PEMs with educational workshops observed minimal differences. Two studies comparing PEMs and education outreach did not have statistically significant differences between the groups. It was not possible to explore potential effect modifiers across studies. The results of this review suggest that when compared to no intervention, PEMs when used alone may have a beneficial effect on process outcomes but not on patient outcomes. Despite this wide of range of effects reported for PEMs, clinical significance of the observed effect sizes is not known. There is insufficient information about how to optimise educational materials. The effectiveness of educational materials compared to other interventions is uncertain.

Clinical practice is not always evidence-based and, therefore, may not optimise patient outcomes. Local opinion leaders (OLs) are individuals perceived as credible and trustworthy, who disseminate and implement best evidence, for instance through informal one-to-one teaching or community outreach education visits. The use of OLs is a promising strategy to bridge evidence-practice gaps. This is an update of a Cochrane review published in 2011. To assess the effectiveness of local opinion leaders to improve healthcare professionals' compliance with evidence-based practice and patient outcomes. We searched CENTRAL, MEDLINE, Embase, three other databases and two trials registers on 3 July 2018, together with searching reference lists of included studies and contacting experts in the field. We considered randomised studies comparing the effects of local opinion leaders, either alone or with a single or more intervention(s) to disseminate evidence-based practice, with no intervention, a single intervention, or the same single or more intervention(s). Eligible studies were those reporting objective measures of professional performance, for example, the percentage of patients being prescribed a specific drug or health outcomes, or both. We included all studies independently of the method used to identify OLs. We used standard Cochrane procedures in this review. The main comparison was (i) between any intervention involving OLs (OLs alone, OLs with a single or more intervention(s)) versus any comparison intervention (no intervention, a single intervention, or the same single or more intervention(s)). We also made four secondary comparisons: ii) OLs alone versus no intervention, iii) OLs alone versus a single intervention, iv) OLs, with a single or more intervention(s) versus the same single or more intervention(s), and v) OLs with a single or more intervention(s) versus no intervention. We included 24 studies, involving more than 337 hospitals, 350 primary care practices, 3005 healthcare professionals, and 29,167 patients (not all studies reported this information). A majority of studies were from North America, and all were conducted in high-income countries. Eighteen of these studies (21 comparisons, 71 compliance outcomes) contributed to the median adjusted risk difference (RD) for the main comparison. The median duration of follow-up was 12 months (range 2 to 30 months). The results suggested that the OL interventions probably improve healthcare professionals' compliance with evidence-based practice (10.8% absolute improvement in compliance, interquartile range (IQR): 3.5% to 14.6%; moderate-certainty evidence).Results for the secondary comparisons also suggested that OLs probably improve compliance with evidence-based practice (moderate-certainty evidence): i) OLs alone versus no intervention: RD (IQR): 9.15% (-0.3% to 15%); ii) OLs alone versus a single intervention: RD (range): 13.8% (12% to 15.5%); iii) OLs, with a single or more intervention(s) versus the same single or more intervention(s): RD (IQR): 7.1% (-1.4% to 19%); iv) OLs with a single or more intervention(s) versus no intervention: RD (IQR):10.25% (0.6% to 15.75%).It is uncertain if OLs alone, or in combination with other intervention(s), may lead to improved patient outcomes (3 studies; 5 dichotomous outcomes) since the certainty of evidence was very low. For two of the secondary comparisons, the IQR included the possibility of a small negative effect of the OL intervention. Possible explanations for the occasional negative effects are, for example, the possibility that the OLs may have prioritised some outcomes, at the expense of others, or that an unaccounted outcome difference at baseline, may have given a faulty impression of a negative effect of the intervention at follow-up. No study reported on costs or cost-effectiveness.We were unable to determine the comparative effectiveness of different approaches to identifying OLs, as most studies used the sociometric method. Nor could we determine which methods used by OLs to educate their peers were most effective, as the methods were poorly described in most studies. In addition, we could not determine whether OL teams were more effective than single OLs. Local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence-based practice, but the effectiveness varies both within and between studies.The effect on patient outcomes is uncertain. The costs and the cost-effectiveness of the intervention(s) is unknown. These results are based on heterogeneous studies differing in types of intervention, setting, and outcomes. In most studies, the role and actions of the OL were not clearly described, and we cannot, therefore, comment on strategies to enhance their effectiveness. It is also not clear whether the methods used to identify OLs are important for their effectiveness, or whether the effect differs if education is delivered by single OLs or by multidisciplinary OL teams. Further research may help us to understand how these factors affect the effectiveness of OLs.

Multiple overlapping systematic reviews facilitate the origin of disputes: the case of thrombolytic therapy for pulmonary embolism

Many experts believe that hospitals with more frequent readmissions provide lower-quality care, but little is known about how the preventability of readmissions might change over the postdischarge time frame. To determine whether readmissions within 7 days of discharge differ from those between 8 and 30 days after discharge with respect to preventability. Prospective cohort study. 10 academic medical centers in the United States. 822 adults readmitted to a general medicine service. For each readmission, 2 site-specific physician adjudicators used a structured survey instrument to determine whether it was preventable and measured other characteristics. Overall, 36.2% of early readmissions versus 23.0% of late readmissions were preventable (median risk difference, 13.0 percentage points [interquartile range, 5.5 to 26.4 percentage points]). Hospitals were identified as better locations for preventing early readmissions (47.2% vs. 25.5%; median risk difference, 22.8 percentage points [interquartile range, 17.9 to 31.8 percentage points]), whereas outpatient clinics (15.2% vs. 6.6%; median risk difference, 10.0 percentage points [interquartile range, 4.6 to 12.2 percentage points]) and home (19.4% vs. 14.0%; median risk difference, 5.6 percentage points [interquartile range, -6.1 to 17.1 percentage points]) were better for preventing late readmissions. Physician adjudicators were not blinded to readmission timing, community hospitals were not included in the study, and readmissions to nonstudy hospitals were not included in the results. Early readmissions were more likely to be preventable and amenable to hospital-based interventions. Late readmissions were less likely to be preventable and were more amenable to ambulatory and home-based interventions. Association of American Medical Colleges.

BackgroundAudit and feedback on professional practice and health care outcomes are the most often used interventions to change behaviour of professionals and improve quality of health care. However, limited information is available regarding preferred feedback for patients, professionals and health insurers.ObjectiveInvestigate the (differences in) preferences of receiving feedback between stakeholders, using the Dutch Head and Neck Audit as an example.MethodsA total of 37 patients, medical specialists, allied health professionals and health insurers were interviewed using semi‐structured interviews. Questions focussed on: “Why,” “On what aspects” and “How” do you prefer to receive feedback on professional practice and health care outcomes?ResultsAll stakeholders mentioned that feedback can improve health care by creating awareness, enabling self‐reflection and reflection on peers or colleagues, and by benchmarking to others. Patients prefer feedback on the actual professional practice that matches the health care received, whereas medical specialists and health insurers are interested mainly in health care outcomes. All stakeholders largely prefer a bar graph. Patients prefer a pie chart for patient‐reported outcomes and experiences, while Kaplan‐Meier survival curves are preferred by medical specialists. Feedback should be simple with firstly an overview, and 1‐4 times a year sent by e‐mail. Finally, patients and health professionals are cautious with regard to transparency of audit data.ConclusionsThis exploratory study shows how feedback preferences differ between stakeholders. Therefore, tailored reports are recommended. Using this information, effects of audit and feedback can be improved by adapting the feedback format and contents to the preferences of stakeholders.

Objectives: With reinterest in lateral extra-articular tenodesis (LET), there is an imperative need to evaluate the rigor of clinical studies regarding LET with anterior cruciate ligament reconstruction (ACLR). This study aims to present a comprehensive picture of the robustness of evidence from comparative studies regarding LET and to inform evidence-based medical decision-making for current practitioners. We hypothesized that the results of these analyses will show statistical fragility, consistent with similar evidence across the orthopaedic literature. Methods: Using the PubMed database, comparative studies and randomized controlled trials (RCTs) related to LET as an augmentation procedure to ACLR or revision ACLR published between 2000 and 2022 were identified and collected. Studies on the topic were broadly queried for relevance, and after screening, they were included if they were (1) pertaining to LET as an augmentation to ACLR or revision ACLR, and (2) designed as comparative studies or RCTs. Exclusion criteria included (1) cadaveric, nonhuman, in vitro, laboratory, or surgical technique (without patient outcomes); (2) commentary, editorial, letter to the editor, conference reports, future study design/published protocol; (3) only abstract available; (4) non-English; or (5) lacked the statistical basis for a fragility analysis. Manuscripts were independently reviewed by three authors and variables of interest of interest were extracted, including both dichotomous and continuous variables relevant to clinical decision making. Discrepancies were resolved via paired discussions. For each dichotomous outcome, we calculated a Fragility Index (FI). FI was calculated using a 2-by-2 contingency table and the Fisher's exact test using the method outlined in Figure 1. For each continuous outcome, we calculated a Continuous Fragility Index (CFI). CFI was calculated using Welch's t-test and the method proposed by Caldwell et al to expand fragility analysis to continuous variables. The analysis for each outcome was conducted with n = 5 simulations using synthetic, representative data generated from the reported sample mean, standard deviation, and sample size for both the experimental and control arms. For both dichotomous and continuous outcomes, the statistical fragility was reported using median and interquartile range. Comparisons of mean statistical fragility were conducted using a nonparametric t test. Data were analyzed using Python 3.7. Results: Out of the 455 initially identified studies, 178 full texts were screened. Ultimately, 29 studies were included in the final analysis, including 20 comparative studies and 9 RCTs. 18 studies reported dichotomous outcomes and 27 studies reported continuous outcomes, which resulted in a cumulative total of 48 dichotomous outcomes and 265 continuous outcomes for analysis. Figure 4 reports the distribution of FI and CFI values for all dichotomous and continuous outcomes. The median FI was 9.0 (25-75th percentile, 7.0-13.25). The median CFI was 7.8 (25-75th percentile, 4.2-19.6). Commonly reported dichotomous outcomes included clinical failure, graft rupture, and return to sport. The number of outcomes in which loss to follow-up (LTF) exceeded the FI was 13 (27%). Reported significant dichotomous outcomes were significantly more fragile than outcomes that were not significant (fragility quotient [FQ] = 0.02 vs 0.11, Welch t test p &lt; 0.001); however, most dichotomous outcomes were reported insignificant (79.2%) (Table 1). Commonly reported continuous outcomes across different studies included KOOS, IKDC, and Lysholm scores. LTF exceeded the CFI for 20 outcomes (8%). In contrast to dichotomous outcomes, continuous outcomes from RCTs and comparative studies were equivalently robust (continuous fragility quotient [CFQ] = 0.12 vs 0.12). The CFQ of both significant and nonsignificant continuous outcomes were similarly equal (0.12 vs 0.12) (Table 2). Reports on fragility quotients from studies that focused on LET for either primary or revision ACLR clinically are presented in Table 3. These subgroups excluded translational research and imaging studies. The FQ and CFQ for studies focused on revision ACLR were larger (0.117 and 0.113) than those focused on primary ACLR (0.042 and 0.095). Conclusions: Research regarding the usage of LET as an augmentation procedure for ACLR and revision ACLR is more statistically robust than many other topics within orthopaedics. Studies regarding LET with primary ACLR are more fragile than those for revision ACLR. Given the resurgent interest in this procedure and the current mixed evidence regarding its effectiveness and indications, we recommend the future reporting of fragility quotients alongside p-values, to assist clinicians in assessing the robustness of new evidence to inform decision making.