P68 Exploiting insights and harnessing heterogeneity: combining intervention component analysis (ICA) and qualitative comparative analysis (QCA) to explore context and implementation in systematic reviews of complex interventions

Abstract

<h3>Background</h3> There is growing acknowledgement that the effectiveness of many health interventions is influenced by complex interplays of intervention characteristics as well as implementation and contextual factors. The treatment of interventions as a ‘black box’ in many systematic reviews has been critiqued for failing to provide review users with vital information. This has led systematic reviewers to search for new methods that are capable of revealing these complexities. <h3>Methods</h3> Intervention Component Analysis (ICA) uses qualitative data analysis techniques and draws on informal experience-based insights from trialists – often reported in the discussion section of published trial reports – about what they felt led to the success of an intervention or what inhibited its success. By exploiting this underutilised information from trial reports ICA seeks to generate a ‘real-world’ understanding of how interventions worked ‘on the ground’. QCA has recently been applied systematic reviews to explore the reasons for variation in outcomes amongst apparently similar interventions. Using set theory it employs a systematic and structured approach to identify the necessary and sufficient conditions for intervention success (or failure). Rather than the variable-oriented approaches used in statistical synthesis approaches, the case-based approach employed in QCA makes it amenable to study intervention complexity as it is able to examine combinations of features and multiple pathways to the same outcome. The case-based approach requires a deep, holistic knowledge of interventions, thus indicating the suitability of combining QCA with ICA. <h3>Results</h3> We draw on several case examples where (ICA) has been successfully combined with Qualitative Comparative Analysis (QCA) in systematic reviews to reveal important combinations of intervention, contextual and implementation features associated with successful interventions. We illustrate how ICA generated new knowledge and insights about potentially important implementation and context features that could moderate the success of interventions. We then demonstrate how QCA was able to substantiate the association between combinations of these features and intervention success. <h3>Discussion</h3> The combination of ICA and QCA offers the benefits of a rich, experience-based understanding of how implementation factors interact with intervention and contextual features with a formal and systematic analysis of the relationship between these factors and intervention effects. In doing so, it reveals the opportunity to make the most of the evidence contained in trial reports, to use robust and systematic approaches for making sense of that information in relation to trial outcomes, and to provide review users with vital details about key intervention and implementation features.