Systematic Reviews Of Complex Interventions Research Articles

Increasing comprehensiveness and reducing workload in a systematic review of complex interventions using automated machine learning.

As part of our ongoing systematic review of complex interventions for the primary prevention of cardiovascular diseases, we have developed and evaluated automated machine-learning classifiers for title and abstract screening. The aim was to develop a high-performing algorithm comparable to human screening. We followed a three-phase process to develop and test an automated machine learning-based classifier for screening potential studies on interventions for primary prevention of cardiovascular disease. We labelled a total of 16,611 articles during the first phase of the project. In the second phase, we used the labelled articles to develop a machine learning-based classifier. After that, we examined the performance of the classifiers in correctly labelling the papers. We evaluated the performance of the five deep-learning models [i.e. parallel convolutional neural network ( CNN ), stacked CNN , parallel-stacked CNN , recurrent neural network ( RNN ) and CNN-RNN]. The models were evaluated using recall, precision and work saved over sampling at no less than 95% recall. We labelled a total of 16,611 articles, of which 676 (4.0%) were tagged as 'relevant' and 15,935 (96%) were tagged as 'irrelevant'. The recall ranged from 51.9% to 96.6%. The precision ranged from 64.6% to 99.1%. The work saved over sampling ranged from 8.9% to as high as 92.1%. The best-performing model was parallel CNN , yielding a 96.4% recall, as well as 99.1% precision, and a potential workload reduction of 89.9%. We used words from the title and the abstract only. More work needs to be done to look into possible changes in performance, such as adding features such as full document text. The approach might also not be able to be used for other complex systematic reviews on different topics. Our study shows that machine learning has the potential to significantly aid the labour-intensive screening of abstracts in systematic reviews of complex interventions. Future research should concentrate on enhancing the classifier system and determining how it can be integrated into the systematic review workflow. This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in Health Technology Assessment. See the NIHR Journals Library website for further project information.

PROTOCOL: Involving men and boys in family planning: A systematic review of the effective components and characteristics of complex interventions in low- and middle-income countries.

PROTOCOL: Involving men and boys in family planning: A systematic review of the effective components and characteristics of complex interventions in low- and middle-income countries.

Conveying Results from Qualitative Comparative Analyses within Systematic Reviews: A Randomized Trial

Research ObjectiveQualitative comparative analysis (QCA) is a supplemental method proposed for synthesizing results from systematic reviews of complex interventions but is unfamiliar to most end users of systematic reviews. In our previous research, end users identified several barriers to understanding QCA results, including presentation formats. The objective of this study was to identify the most effective presentation approach to communicating results from a QCA within the context of a systematic review.Study DesignRandomized experiment. We provided all participants with information about a hypothetical systematic review of a complex intervention for a common health condition then randomized each participant to review the results of the QCA in 1 of 3 presentation formats (text, table, or graphic). Each format contained the same results information. We then assessed ease of understanding (ie, subjective comprehension) and objective comprehension of the QCA results with an online questionnaire.Population Studied254 clinicians, public health practitioners, policy makers, and health researchers who had read at least 1 systematic review within the previous 5 years that we recruited via email from U.S.‐ and U.K.‐based public health, health care, and academic organizations.Principal FindingsWe found a statistically significant difference (P = .035) across the 3 presentation formats in subjective comprehension as assessed by an index we created from 6 items (Cronbach’s alpha = 0.89). The mean subjective comprehension was highest for the graphic format (2.61 [SD 0.56]) followed by text (2.57 [SD 0.55]) and table (2.40 [SD 0.55])) on a scale of 1 (poor comprehension) to 4 (good comprehension). In preplanned comparisons, participants who were randomized to the graphic format scored higher compared to the table (P = .018), and participants allocated to text format scored higher compared to table (P = .034), but these differences were not considered statistically significant based on a Bonferroni‐adjusted P value of .0167.We assessed objective comprehension with 7 items designed to assess parameter interpretation and 7 items designed to assess configuration interpretation. The mean number of correct parameter items was 4.04 (SD 1.84), and the mean number of correct configuration items was 4.61 (SD 1.36). No significant differences across presentation formats were observed across groups with regard to objective comprehension overall (P = .071). However, we identified a statistically significant difference across groups for parameter interpretation (P = .001) but not for configuration interpretation (P = .090). For parameter interpretation, scores among participants that received the graphic format were significantly higher (4.70) than scores for participants who received the text (mean 3.88, P = .001) or table (mean 3.67, P = .004) formats.Study limitations include the use of a convenience sample, evaluation of the different formats using only 1 example systematic review, and participants did not evaluate one format against another.ConclusionsA table may be the least effective format for presenting results related to a QCA within a systematic review of a complex intervention. A graphic format appears to be superior on some measures of objective comprehension.Implications for Policy or PracticeResearchers using QCA within a systematic review should be aware of differences in reader understanding based on presentation format and should consider communicating results with a graphic when possible.Primary Funding SourceAgency for Healthcare Research and Quality.

Assessing the Effectiveness of Complex Interventions to Meet the Needs of VA Stakeholders: Experience of the Department of Veterans Affairs Evidence Synthesis Program.

Background:Complex health care interventions involve multiple distinct elements that contribute to their functioning. Conducting systematic reviews of complex interventions has substantial challenges. Although methodological guidance exists, less is known about the practical strategies and approaches undertaken by systematic review groups to navigate common challenges and enhance impacts of systematic review findings.Objectives:Describe pragmatic approaches taken by Department of Veterans Affairs Evidence Synthesis Programs (VA ESP) in conducting systematic reviews of effectiveness and implementation barriers and facilitators for complex interventions to provide VA stakeholders with evidence to guide national health care practice and policy.Results:We describe 3 systematic reviews conducted by VA ESP teams to evaluate the evidence for complex health care interventions. We summarize key findings, implications for future research needs and policy, dissemination of findings, and approaches taken to address common challenges. The VA ESP experience adds to existing systematic review methods and provides a perspective on generating rigorous and relevant reviews of complex interventions.Conclusions:Reviews of complex interventions often encounter challenges related to sources of variability in many dimensions, and lack of clarity and information in reporting of intervention elements, local context, and implementation factors. Evidence synthesis teams should work closely with stakeholders to understand their needs, synthesize and interpret results in meaningful ways, and explore implications that are most relevant for day-to-day clinical practice and operational decisions of learning health care systems. More evaluation of the impact of systematic reviews may improve uptake of findings from future reviews and enhance translation of evidence into practice and policy.

Towards greater understanding of implementation during systematic reviews of complex healthcare interventions: the framework for implementation transferability applicability reporting (FITAR)

BackgroundThere have been calls for greater consideration of applicability and transferability in systematic reviews, to improve their usefulness in informing policy and practice. Understanding how evidence is, or is not applicable and transferable to varying local situations and contexts, is a key challenge for systematic review synthesis in healthcare. Assessing applicability and transferability in systematic reviews is reported to be difficult, particularly in reviews of complex interventions. There is a need for exploration of factors perceived to be important by policy-makers, and for further guidance on which items should be reported. In this paper we focus on the process of development of a framework that can be used by systematic reviewers to identify and report data across studies relating to applicability and transferability.MethodsThe framework was developed by scrutinising existing literature on applicability and transferability, examining data during a systematic review of highly complex changes to health service delivery, and was informed by stakeholder engagement. The items of the framework were thus grounded in both data identified during a real review, and stakeholder input. The paper describes examples of data identified using the framework during a review of integrated care interventions, and outlines how it informed analysis and reporting of the review findings.ResultsThe Framework for Implementation Transferability Applicability Reporting (FITAR) comprises 44 items which can be used to structure analysis and reporting across studies during systematic reviews of complex interventions. The framework prompts detailed consideration of contextual data during extraction and reporting, within areas of: patient type and populations; type of organisations and systems; financial and commissioning processes; systems leadership elements; features of services; features of the workforce; and finally elements of the interventions/initiatives.ConclusionsUse of the framework during our review of complex healthcare interventions helped the review team to surface contextual data, which may not be commonly extracted, analysed and reported. Further exploration and evaluation of systems for identifying and reporting these factors during reviews is required.

Synthesising quantitative evidence in systematic reviews of complex health interventions

Public health and health service interventions are typically complex: they are multifaceted, with impacts at multiple levels and on multiple stakeholders. Systematic reviews evaluating the effects of complex health interventions...

AHRQ series on complex intervention systematic reviews—paper 3: adapting frameworks to develop protocols

BackgroundOnce a proposed topic has been identified for a systematic review and has undergone a question formulation stage, a protocol must be developed that specifies the scope and research questions in detail and outlines the methodology for conducting the systematic review. RationaleFramework modifications are often needed to accommodate increased complexity. We describe and give examples of adaptations and alternatives to traditional analytic frameworks. DiscussionThis article identifies and describes elements of frameworks and how they can be adapted to inform the protocol and conduct of systematic reviews of complex interventions. Modifications may be needed to adapt the population, intervention, comparators, and outcomes normally used in protocol development to successfully describe complex interventions; in some instances, alternative frameworks may be better suited. Possible approaches to analytic frameworks for complex interventions that illustrate causal and associative linkages are outlined, including time elements, which systematic reviews of complex interventions may need to address. The need for and specifics of the accommodations vary with details of a specific systematic review. This in turn helps determine whether traditional frameworks are sufficient, can be refined, or if alternate frameworks must be adopted.

AHRQ series on complex intervention systematic reviews—paper 2: defining complexity, formulating scope, and questions

BackgroundThe early stages of a systematic review set the scope and expectations. This can be particularly challenging for complex interventions given their multidimensional and dynamic nature. RationaleThis paper builds on concepts introduced in paper 1 of this series. It describes the methodological, practical, and philosophical challenges and potential approaches for formulating the questions and scope of systematic reviews of complex interventions. Furthermore, it discusses the use of theory to help organize reviews of complex interventions. DiscussionMany interventions in medicine, public health, education, social services, behavioral health, and community programs are complex, and they may not fit neatly within the established paradigm for reviews of straightforward interventions. This paper provides conceptual and operational guidance for these early stages of scope formulation to assist authors of systematic reviews of complex interventions.

AHRQ series on complex intervention systematic reviews—paper 7: PRISMA-CI elaboration and explanation

BackgroundComplex interventions are widely used in health care, public health, education, criminology, social work, business, and welfare. They have increasingly become the subject of systematic reviews and are challenging to effectively report. The Complex Interventions Methods Workgroup developed an extension to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Complex Interventions (PRISMA-CI). RationaleFollowing the EQUATOR Network guidance for Preferred Reporting Items for Systematic Reviews and Meta-Analysis extensions, this Explanation and Elaboration (EE) document accompanies the PRISMA-CI checklist to promote consistency in reporting of systematic reviews of complex interventions. DiscussionsThe EE document explains the meaning and rationale for each unique PRISMA-CI checklist item and provides examples to assist systematic review authors in operationalizing PRISMA-CI guidance. The Complex Interventions Workgroup developed PRISMA-CI as an important start toward increased consistency in reporting of systematic reviews of complex interventions. Because the field is rapidly expanding, the Complex Interventions Methods Workgroup plans to re-evaluate periodically for the need to add increasing specificity and examples as the field matures.

AHRQ series on complex intervention systematic reviews—paper 6: PRISMA-CI extension statement and checklist

BackgroundComplex interventions are widely used in health systems, public health, education, and communities and are increasingly the subject of systematic reviews. Oversimplification and inconsistencies in reporting about complex interventions can limit the usability of review findings. RationaleAlthough guidance exists to ensure that reports of individual studies and systematic reviews adhere to accepted scientific standards, their design-specific focus leaves important reporting gaps relative to complex interventions in health care. This paper provides a stand-alone extension to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting tool for complex interventions—PRISMA-CI—to help authors, publishers, and readers understand and apply to systematic reviews of complex interventions. DiscussionPRISMA-CI development followed the Enhancing the QUAlity and Transparency Of health Research Network guidance for extensions and focused on adding or modifying only essential items that are truly unique to complex interventions and are not covered by broader interpretation of current PRISMA guidance. PRISMA-CI provides an important structure and guidance for systematic reviews and meta-analyses for the highly prevalent and dynamic field of complex interventions.

Lay-led and peer support interventions for adolescents with asthma.

Adolescents with asthma are at high risk of poor adherence with treatment. This may be compounded by activities that worsen asthma, in particular smoking. Additional support above and beyond routine care has the potential to encourage good self-management. We wanted to find out whether sessions led by their peers or by lay leaders help to reduce these risks and improve asthma outcomes among adolescents. To assess the safety and efficacy of lay-led and peer support interventions for adolescents with asthma. We identified trials from the Cochrane Airways Trials Register, which contains reports of randomised trials obtained from multiple electronic and handsearched sources, and we searched trial registries and reference lists of primary studies. We conducted the most recent searches on 25 November 2016. Eligible studies randomised adolescents with asthma to an intervention led by lay people or peers or to a control. We included parallel randomised controlled trials with individual or cluster designs. We included studies reported as full text, those published as abstract only and unpublished data. Two review authors screened the searches, extracted numerical data and study characteristics and assessed each included study for risk of bias. Primary outcomes were asthma-related quality of life and exacerbations requiring at least a course of oral steroids. We graded the analyses and presented evidence in a 'Summary of findings' table.We analysed dichotomous data as odds ratios, and continuous data as mean differences (MD) or standardised mean differences, all with a random-effects model. We assessed clinical, methodological and statistical heterogeneity when performing meta-analyses, and we described skewed data narratively. Five studies including a total of 1146 participants met the inclusion criteria for this review. As ever with systematic reviews of complex interventions, studies varied by design (cluster and individually randomised), duration (2.5 to 9 months), setting (school, day camp, primary care) and intervention content. Most risk of bias concerns were related to blinding and incomplete reporting, which limited the meta-analyses that could be performed. Studies generally controlled well for selection and attrition biases.All participants were between 11 and 17 years of age. Asthma diagnosis and severity varied, as did smoking prevalence. Three studies used the Triple A programme; one of these studies tested the addition of a smoke-free pledge; another delivered peer support group sessions and mp3 messaging to encourage adherence; and the third compared a peer-led asthma day camp with an equivalent camp led by healthcare practitioners.We had low confidence in all findings owing to risk of bias, inconsistency and imprecision. Results from an analysis of asthma-related quality of life based on the prespecified random-effects model were imprecise and showed no differences (MD 0.40, 95% confidence interval (CI) -0.02 to 0.81); a sensitivity analysis based on a fixed-effect model and a responder analysis suggested small benefit may be derived for this outcome. Most other results were summarised narratively and did not show an important benefit of the intervention; studies provided no analysable data on asthma exacerbations or unscheduled visits (data were skewed), and one study measuring adherence reported a drop in both groups. Effects on asthma control favoured the intervention but findings were not statistically significant. Results from two studies with high levels of baseline smoking showed some promise for self-efficacy to stop smoking, but overall nicotine dependence and smoking-related knowledge were not significantly better in the intervention group. Investigators did not report adverse events. Although weak evidence suggests that lay-led and peer support interventions could lead to a small improvement in asthma-related quality of life for adolescents, benefits for asthma control, exacerbations and medication adherence remain unproven. Current evidence is insufficient to reveal whether routine use of lay-led or peer support programmes is beneficial for adolescents receiving asthma care.Ongoing and future research may help to identify target populations for lay-led and peer support interventions, along with attributes that constitute a successful programme.

Adding non-randomised studies to a Cochrane review brings complementary information for healthcare stakeholders: an augmented systematic review and meta-analysis.

BackgroundTo reduce the burden of asthma, chronic disease management (CDM) programmes have been widely implemented and evaluated. Reviews including randomised controlled trials (RCTs) suggest that CDM programmes for asthma are effective. Other study designs are however often used for pragmatic reasons, but excluded from these reviews because of their design. We aimed to examine what complementary information could be retrieved from the addition of non-randomised studies to the studies included in a published Cochrane review on asthma CDM programmes, for healthcare stakeholders involved in the development, implementation, conduct or long-term sustainability of such programmes.MethodsExtending a previously published Cochrane review, we performed a systematic review (augmented review) including any type of study designs instead of only those initially accepted by Cochrane and the Effective Practice and Organization of Care Review group. After double data selection and extraction, we compared study and intervention characteristics, assessed methodological quality and ran meta-analyses, by study design.ResultsWe added 37 studies to the 20 studies included in the Cochrane review. The applicability of results was increased because of the larger variety of settings and asthma population considered. Also, adding non-randomised studies provided new evidence of improvements associated with CDM intervention (i.e. healthcare utilisation, days off work, use of action plan). Finally, evidence of CDM effectiveness in the added studies was consistent with the Cochrane review in terms of direction of effects.ConclusionsThe evidence of this augmented review is applicable to a broader set of patients and settings than those in the original Cochrane review. It also strengthens the message that CDM programmes have a beneficial effect on quality of life and disease severity, meaningful outcomes for the everyday life of patients with asthma. Despite the moderate to low methodological quality of all studies included, calling for caution in results interpretation and improvements in CDM evaluation methods and reporting, the inclusion of a broader set of study designs in systematic reviews of complex interventions, such as chronic disease management, is likely to be of high value and interest to patients, policymakers and other healthcare stakeholders.Electronic supplementary materialThe online version of this article (doi:10.1186/s12913-016-1816-5) contains supplementary material, which is available to authorized users.

Using Qualitative Research to Overcome the Shortcomings of Systematic Reviews When Designing of a Self-Management Intervention for Advanced Cancer Pain

Objectives: To identify the key components for a self-management intervention for advanced cancer pain using evidence drawn from systematic reviews of complex interventions and syntheses of qualitative research. Methods: Evidence from up-to-date systematic reviews was prioritized. Searches were initially undertaken to identify the systematic reviews of effectiveness in Cinahl, Medline, Embase, PsycInfo, and the Cochrane Database of systematic reviews from 2009 to June 2014, using validated search terms. Subsequent searches to identify the qualitative systematic reviews were undertaken in Cinahl, Medline, Embase, and PsycInfo from 2009 to January 2015. The results of the two sets of reviews were integrated using methods based on constant comparative techniques. Results: Four systematic reviews examining interventions for the self-management of advanced cancer pain were identified. Although each review recommended some attributes of a pain management intervention, it was not possible to determine the essential key components. Subsequent searches for qualitative evidence syntheses identified three reviews. These were integrated with the effectiveness reviews. The integration identified key components for a self-management intervention including individualized approaches to care, the importance of addressing patients’ knowledge, skills, and attitudes toward pain management, and the significance of team approaches and inter-disciplinary working in the management of pain. Conclusion: Implementing the findings from systematic reviews of complex interventions is often hindered by a lack of understanding of important contextual components of care, often provided by qualitative research. Using both types of data to provide answers for practice demonstrates the benefits of incorporating qualitative research in reviews of complex interventions by ensuring the strengths of qualitative and quantitative research are combined and that their respective weaknesses are overcome.

Outcomes in systematic reviews of complex interventions never reached “high” GRADE ratings when compared with those of simple interventions

ObjectivesTo investigate the application of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach and the quality of evidence ratings in systematic reviews of complex interventions. Study Design and SettingThis study examined all 40 systematic reviews published in three Cochrane Review Groups from 2013 to May 2014: Cochrane Developmental, Psychosocial and Learning Problems Group (CDPLPG); Cochrane Public Health Group (CPHG); and Cochrane Depression, Anxiety, and Neurosis Group (CCDAN). The reviews were coded and classified into “complex” (n = 24) and “simple” (n = 16) intervention review groups based on the predefined complexity dimensions from the extant literature mapped into the PICOTS framework. All the data were analyzed in these two groups to help identify specific patterns of the GRADE ratings in the reviews of complex interventions. ResultsOutcomes of complex intervention reviews had higher proportions of “very low” quality of evidence ratings compared with those of simple intervention reviews (37.5% vs. 9.1% for the primary benefit outcomes) and were more frequently downgraded for inconsistency, performance bias, and study design. None of the outcomes of complex intervention reviews (0%) were given “high” GRADE ratings. ConclusionResults suggest that the GRADE assessment may not adequately describe the evidence base of complex interventions.

Current use was established and Cochrane guidance on selection of social theories for systematic reviews of complex interventions was developed

ObjectiveTo identify examples of how social theories are used in systematic reviews of complex interventions to inform production of Cochrane guidance. Study Design and SettingSecondary analysis of published/unpublished examples of theories of social phenomena for use in reviews of complex interventions identified through scoping searches, engagement with key authors and methodologists supplemented by snowballing and reference searching. Theories were classified (low-level, mid-range, grand). ResultsOver 100 theories were identified with evidence of proliferation over the last 5 years. New low-level theories (tools, taxonomies, etc) have been developed for classifying and reporting complex interventions. Numerous mid-range theories are used; one example demonstrated how control theory had changed the review's findings. Review-specific logic models are increasingly used, but these can be challenging to develop. New low-level and mid-range psychological theories of behavior change are evolving. No reviews using grand theory (e.g., feminist theory) were identified. We produced a searchable Wiki, Mendeley Inventory, and Cochrane guidance. ConclusionsUse of low-level theory is common and evolving; incorporation of mid-range theory is still the exception rather than the norm. Methodological work is needed to evaluate the contribution of theory. Choice of theory reflects personal preference; application of theory is a skilled endeavor.

How stakeholder participation can contribute to systematic reviews of complex interventions

Although patient and public involvement in research is a requirement for research funding in many countries, the knowledge base for how to effectively involve people—and evidence of the effectiveness of...

Users identified challenges in applying GRADE to complex interventions and suggested an extension to GRADE

ObjectivesTo explore user perspectives on applying the Grading of Recommendations Assessment, Development and Evaluation (GRADE) to systematic reviews of complex interventions. Study Design and SettingThirty-three authors of recent (2013 onward) systematic reviews were contacted regarding their perspectives on using GRADE from three Cochrane review groups: Cochrane Developmental, Psychosocial, and Learning Problems Group; Cochrane Public Health Group; and Cochrane Depression, Anxiety, and Neurosis Group. Framework Analysis was applied to the data to identify the challenges in applying GRADE and suggestions for its extension, that is, adaptation. These two themes were cross-compared between the groups of answers from “simple” vs. “complex” intervention review authors to identify the specific perspectives on using GRADE in reviews of complex interventions. ResultsSpecific challenges were identified in applying GRADE to reviews of complex interventions. These were related to the assessment of nonrandomized studies and performance bias in GRADE. Authors perceived these challenges to contribute to frequent downgrading of the “best evidence possible” for complex interventions. Meanwhile, GRADE was found to lack an analytic approach to enable adequate evidence synthesis and assessment of intervention implementation elements. ConclusionUsers suggest that the GRADE guidance be extended to address-specific considerations for complex interventions.

Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth.

BackgroundSeveral papers report deficiencies in the reporting of information about the implementation of interventions in clinical trials. Information about implementation is also required in systematic reviews of complex interventions to facilitate the translation and uptake of evidence of provider-based prevention and treatment programs. To capture whether and how implementation is assessed within systematic effectiveness reviews, we developed a checklist for implementation (Ch-IMP) and piloted it in a cohort of reviews on provider-based prevention and treatment interventions for children and young people. This paper reports on the inter-rater reliability, feasibility and reasons for discrepant ratings.MethodsChecklist domains were informed by a framework for program theory; items within domains were generated from a literature review. The checklist was pilot-tested on a cohort of 27 effectiveness reviews targeting children and youth. Two raters independently extracted information on 47 items. Inter-rater reliability was evaluated using percentage agreement and unweighted kappa coefficients. Reasons for discrepant ratings were content analysed.ResultsKappa coefficients ranged from 0.37 to 1.00 and were not influenced by one-sided bias. Most kappa values were classified as excellent (n = 20) or good (n = 17) with a few items categorised as fair (n = 7) or poor (n = 1). Prevalence-adjusted kappa coefficients indicate good or excellent agreement for all but one item. Four areas contributed to scoring discrepancies: 1) clarity or sufficiency of information provided in the review; 2) information missed in the review; 3) issues encountered with the tool; and 4) issues encountered at the review level. Use of the tool demands time investment and it requires adjustment to improve its feasibility for wider use.ConclusionsThe case of provider-based prevention and treatment interventions showed relevancy in developing and piloting the Ch-IMP as a useful tool for assessing the extent to which systematic reviews assess the quality of implementation. The checklist could be used by authors and editors to improve the quality of systematic reviews, and shows promise as a pedagogical tool to facilitate the extraction and reporting of implementation characteristics.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-015-0037-7) contains supplementary material, which is available to authorized users.

Seeing the forests and the trees--innovative approaches to exploring heterogeneity in systematic reviews of complex interventions to enhance health system decision-making: a protocol.

BackgroundTo improve quality of care and patient outcomes, health system decision-makers need to identify and implement effective interventions. An increasing number of systematic reviews document the effects of quality improvement programs to assist decision-makers in developing new initiatives. However, limitations in the reporting of primary studies and current meta-analysis methods (including approaches for exploring heterogeneity) reduce the utility of existing syntheses for health system decision-makers. This study will explore the role of innovative meta-analysis approaches and the added value of enriched and updated data for increasing the utility of systematic reviews of complex interventions.Methods/DesignWe will use the dataset from our recent systematic review of 142 randomized trials of diabetes quality improvement programs to evaluate novel approaches for exploring heterogeneity. These will include exploratory methods, such as multivariate meta-regression analyses and all-subsets combinatorial meta-analysis. We will then update our systematic review to include new trials and enrich the dataset by surveying authors of all included trials. In doing so, we will explore the impact of variables not, reported in previous publications, such as details of study context, on the effectiveness of the intervention. We will use innovative analytical methods on the enriched and updated dataset to identify key success factors in the implementation of quality improvement interventions for diabetes. Decision-makers will be involved throughout to help identify and prioritize variables to be explored and to aid in the interpretation and dissemination of results.DiscussionThis study will inform future systematic reviews of complex interventions and describe the value of enriching and updating data for exploring heterogeneity in meta-analysis. It will also result in an updated comprehensive systematic review of diabetes quality improvement interventions that will be useful to health system decision-makers in developing interventions to improve outcomes for people with diabetes.Systematic review registrationPROSPERO registration no. CRD42013005165

Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions.

BackgroundThere is increasing interest in innovative methods to carry out systematic reviews of complex interventions. Theory-based approaches, such as logic models, have been suggested as a means of providing additional insights beyond that obtained via conventional review methods.MethodsThis paper reports the use of an innovative method which combines systematic review processes with logic model techniques to synthesise a broad range of literature. The potential value of the model produced was explored with stakeholders.ResultsThe review identified 295 papers that met the inclusion criteria. The papers consisted of 141 intervention studies and 154 non-intervention quantitative and qualitative articles. A logic model was systematically built from these studies. The model outlines interventions, short term outcomes, moderating and mediating factors and long term demand management outcomes and impacts. Interventions were grouped into typologies of practitioner education, process change, system change, and patient intervention. Short-term outcomes identified that may result from these interventions were changed physician or patient knowledge, beliefs or attitudes and also interventions related to changed doctor-patient interaction. A range of factors which may influence whether these outcomes lead to long term change were detailed. Demand management outcomes and intended impacts included content of referral, rate of referral, and doctor or patient satisfaction.ConclusionsThe logic model details evidence and assumptions underpinning the complex pathway from interventions to demand management impact. The method offers a useful addition to systematic review methodologies.Trial registration numberPROSPERO registration number: CRD42013004037.