Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study.

Simon Lewin,Audrey Rankin,Cathal A. Cadogan,Carmel M. Hughes

doi:10.12688/hrbopenres.13044.1

Abstract

Background: The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews. Worked examples of the tool's application are needed to promote its use and refinement. The aim of this case study was to apply the iCAT_SR to a subset of 20 studies included in a Cochrane review of interventions aimed at improving appropriate polypharmacy in older people. Methods: Interventions were assessed independently by two authors using the six core iCAT_SR dimensions: (1) 'Target organisational levels/categories'; (2) 'Target behaviour/actions'; (3) 'Active intervention components'; (4) 'Degree of tailoring'; (5) 'Level of skill required by intervention deliverers'; (6) 'Level of skill required by intervention recipients'. Attempts were made to apply four optional dimensions: 'Interaction between intervention components'; 'Context/setting'; 'Recipient/provider factors'; 'Nature of causal pathway'. Inter-rater reliability was assessed using Cohen's Kappa coefficient. Disagreements were resolved by consensus discussion. The findings are presented narratively. Results: Assessments involving the core iCAT_SR dimensions showed limited consistency in intervention complexity across included studies, even when categorised according to clinical setting. Interventions were delivered across various organisational levels and categories (i.e. healthcare professionals and patients) and typically comprised multiple components. Intermediate skill levels were required by those delivering and receiving the interventions across all studies. A lack of detail in study reports precluded application of the iCAT_SR's optional dimensions. The inter-rater reliability was substantial (Cohen's Kappa = 0.75) Conclusions: This study describes the application of the iCAT_SR to studies included in a Cochrane systematic review. Future intervention studies need to ensure more detailed reporting of interventions, context and the causal pathways underlying intervention effects to allow a more holistic understanding of intervention complexity and facilitate replication in other settings. The experience gained has helped to refine the original guidance document relating to the application of iCAT_SR.

Highlights

Several definitions of the term ‘complex intervention’ exist in the literature
We report on our experiences of applying intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) to included studies and outline potential refinements to the tool to facilitate its future application as part of systematic reviews
A convenience sample of intervention studies included in a recent update of a Cochrane review of interventions to improve appropriate polypharmacy in older people[12] was assessed using the iCAT_SR6

Summary

Introduction

Several definitions of the term ‘complex intervention’ exist in the literature. For example, the UK Medical Research Council has defined ‘complex interventions’ as “interventions that contain several interacting components” while recognising that there are multiple dimensions of complexity (e.g. the number of intervention components, interactions between components, target behaviours, target groups or organisational levels, outcomes and degree of tailoring permitted)[1]. Interventions targeting key healthcare issues, such as multimorbidity and the use of multiple medications (i.e. polypharmacy) in older people, are often complex in that the interventions typically involve multiple components[4,5]. This broad categorisation of interventions based on their components limits our potential to systematically compare interventions and their effects. Results: Assessments involving the core iCAT_SR dimensions showed limited consistency in intervention complexity across included studies, even when categorised according to clinical setting. Future intervention studies need to ensure more detailed reporting of interventions, context and the causal pathways underlying intervention effects to allow a more holistic version 1

Objectives

Methods

Results

Discussion

Conclusion