Trial Of Complex Intervention Research Articles

In response to Spillane V., Byrne M.C., Byrne M., Leathem C.S., O’Malley M. & Cupples M.E. (2007) Monitoring treatment fidelity in a randomized trial of a complex intervention. Journal of Advanced Nursing60(3), 343–352.

Spillane et al. (2007) outline in their JAN paper the methods they used to monitor (and presumably promote) intervention fidelity within a trial of a ‘complex’ intervention to encourage medication adherence and lifestyle change among heart disease patients. The study involved 48 general practices. The authors misquote our own work in complex, multi-site cluster-randomized trials. Given that the point is germane to their argument, we would like to correct this. We are cited as saying that trials of complex interventions ‘must strive to consistently implement the same intervention by standardizing its content and delivery’. Actually what we said was that something has to be standardized in a complex intervention, for the sake of both internal and external validity (Hawe et al. 2004), but it does not have to be the content of the intervention, nor the way it is delivered. In our paper, we lamented the tendency for investigators to feel that the form of an intervention had to remain the same in every site, mistakenly believing that this was an essential requirement of a randomized trial. We felt that this over-controlled the intervention. In some cases, standardizing interventions by form (e.g. using exactly the same patient education materials in every site) might be going too far in ‘treatment fidelity’, when tailoring to context might be more effective (e.g. allowing materials to be adapted to local cultural styles and literacy levels). In the latter case, the function of an intervention component would remain the same, but the form could be different in different sites. We argued that standardizing by the function that a component plays in an intervention, instead of standardizing by the form it takes, may be more appropriate in many complex interventions. Indeed, the more complex an intervention becomes, the more it is necessary to have rigorous theory about the process and principles of the change process being tested, but to be flexible about the form that this takes in each site. Anyone versed in fields like active listening or community development, for example, would appreciate that it is impossible to prescribe the exact form the intervention takes, but the process, principles and critical sequences ought to be recognizable and replicable. Thus, we would suggest that both the content and the delivery of Spillane’s SPHERE intervention could differ among the 24 interventions sites, but the integrity of the intervention could be fully preserved if the principles of training, delivery and enactment of skills (as they have specified) coincide with the theory the intervention is testing.

Bench-to-bedside review: The evaluation of complex interventions in critical care

Complex interventions, such as the introduction of medical emergency teams or an early goal-directed therapy protocol, are developed from a number of components that may act both independently and inter-dependently. There is an emerging body of literature advocating the use of integrated complex interventions to optimise the treatment of critically ill patients. As with any other treatment, complex interventions should undergo careful evaluation prior to widespread introduction into clinical practice. During the development of an international collaboration of researchers investigating protocol-based approaches to the resuscitation of patients with severe sepsis, we examined the specific issues related to the evaluation of complex interventions. This review outlines some of these issues. The issues specific to trials of complex interventions that require particular attention include determining an appropriate study population and defining current treatments and outcomes in that population, defining the study intervention and the treatment to be used in the control group, and deploying the intervention in a standardised manner. The context in which the research takes place, including existing staffing levels and existing protocols and procedures, is crucial. We also discuss specific details of trial execution, in particular randomization, blinded outcome adjudication and analysis of the results, which are key to avoiding bias in the design and interpretation of such trials.These aspects of study design impact upon the evaluation of complex interventions in critical care. Clinicians should also consider these specific issues when implementing new complex interventions into their practice.

Monitoring treatment fidelity in a randomized controlled trial of a complex intervention

This paper is a report of a study to describe how treatment fidelity is being enhanced and monitored, using a model from the National Institutes of Health Behavior Change Consortium. The objective of treatment fidelity is to minimize errors in interpreting research trial outcomes, and to ascribe those outcomes directly to the intervention at hand. Treatment fidelity procedures are included in trials of complex interventions to account for inferences made from study outcomes. Monitoring treatment fidelity can help improve study design, maximize reliability of results, increase statistical power, determine whether theory-based interventions are responsible for observed changes, and inform the research dissemination process. Treatment fidelity recommendations from the Behavior Change Consortium were applied to the SPHERE study (Secondary Prevention of Heart DiseasE in GeneRal PracticE), a randomized controlled trial of a complex intervention. Procedures to enhance and monitor intervention implementation included standardizing training sessions, observing intervention consultations, structuring patient recall systems, and using written practice and patient care plans. The research nurse plays an important role in monitoring intervention implementation. Several methods of applying treatment fidelity procedures to monitoring interventions are possible. The procedure used may be determined by availability of appropriate personnel, fiscal constraints, or time limits. Complex interventions are not straightforward and necessitate a monitoring process at trial stage. The Behavior Change Consortium's model of treatment fidelity is useful for structuring a system to monitor the implementation of a complex intervention, and helps to increase the reliability and validity of evaluation findings.

Understanding the implementation of complex interventions in health care: the normalization process model

BackgroundThe Normalization Process Model is a theoretical model that assists in explaining the processes by which complex interventions become routinely embedded in health care practice. It offers a framework for process evaluation and also for comparative studies of complex interventions. It focuses on the factors that promote or inhibit the routine embedding of complex interventions in health care practice.MethodsA formal theory structure is used to define the model, and its internal causal relations and mechanisms. The model is broken down to show that it is consistent and adequate in generating accurate description, systematic explanation, and the production of rational knowledge claims about the workability and integration of complex interventions.ResultsThe model explains the normalization of complex interventions by reference to four factors demonstrated to promote or inhibit the operationalization and embedding of complex interventions (interactional workability, relational integration, skill-set workability, and contextual integration).ConclusionThe model is consistent and adequate. Repeated calls for theoretically sound process evaluations in randomized controlled trials of complex interventions, and policy-makers who call for a proper understanding of implementation processes, emphasize the value of conceptual tools like the Normalization Process Model.

Methods to recruit and retain a cohort of young-adult injection drug users for the Third Collaborative Injection Drug Users Study/Drug Users Intervention Trial (CIDUS III/DUIT)

Background New injection drug users (IDUs) are at high risk for blood-borne viral infections. Given U.S. policy to only fund proven-effective HIV prevention interventions, insights into conducting intervention trials among young IDUs are provided here by describing methods and participants’ characteristics in the CIDUS III/DUIT study. Methods In 2002–2004, 15–30-year-old IDUs in Baltimore, Chicago, Los Angeles, New York, and Seattle were recruited through community outreach, advertising and coupon-based participant referrals. Baseline interviews assessed sociodemographics, injection, and sexual behaviors. Antibody tests for HIV and hepatitis A, B, and C viruses (HAV, HBV, and HCV) were conducted. IDUs who were HIV and HCV antibody negative at baseline were eligible to participate in a randomized controlled HIV/HCV prevention trial. Follow-up assessments were conducted 3 and 6 months post-intervention. Data were analyzed to identify participant differences at baseline by city, trial enrollment, and trial retention. Results Baseline assessments were completed by 3285 IDUs. Participants were mean age 23.8 years, 69% male, 64% White, 17% Hispanic, and 8% Black. Seroprevalence of HIV, HCV, HBV, and HAV antibodies were 2.9, 34.4, 22.4, and 19.3%, respectively. Of the 2062 (62.7%) baseline participants who were HIV and HCV antibody negative, 859 (41.7%) were randomized. At least one follow-up assessment was completed by 712 (83%) randomized participants. Contextual factors, primarily homelessness, were associated with lower enrollment and retention. Conclusions Recruitment and retention of young-adult IDUs for complex intervention trials is complicated, yet feasible. Risk behaviors among participants enrolling in and completing the trial reflected those eligible to enroll.

Modelling treatment‐effect heterogeneity in randomized controlled trials of complex interventions (psychological treatments)

We describe instrumental variable (IV) methods for the estimation of the 'dose'-response effects of psychological interventions in randomized controlled trials in which there is variability in the patients' adherence to the allocated therapy (that is, variability in the actual number of sessions of therapy attended) and also variability in the strength of the therapeutic alliance between patients and their therapists. The effect of the therapy on outcome is assumed to be a function of both the number of sessions attended and the strength of the therapeutic alliance, with no intervention effects in the absence of any sessions attended (an exclusion restriction) and the effect of the strength of the alliance being represented by a multiplicative term (interaction) in the treatment-effect model. The IV methods that are described allow for: (a) hidden confounding between sessions, alliance and outcome; (b) measurement errors in the alliance; and (c) that alliance is only measured in those receiving treatment. Three two-stage estimation procedures are illustrated, and their equivalence demonstrated, through Monte Carlo simulation and analysis of the results of an actual trial.

Improving the quality of mental health care in primary care settings: a view from the United Kingdom

In the forty years since 'general practice' became a focus for research in psychiatry the UK there have been considerable developments in policy, practice and research. The aim of this paper is to review recent research and policy developments concerned with improving quality of mental health in primary care settings. Methods: Narrative review of the literature. Results: Disappointing results from large scales trials in the last decade have led to a move towards more exploratory studies and attempts to understand more about contextual factors. Policy initiatives such as the NICE (National Institute of Health and Clinical Excellence) guidelines have set clear standards for the delivery of care, but considerable variation in quality of care persists in primary care settings. The Medical Research Council of the UK has suggested a sequential model for future randomised trials of complex interventions. Conclusion: Major outstanding challenges are the difficulties in recruiting GPs (General Practitioners) into research studies who are not particularly interested in mental health and linking research and policy such that the findings of such studies are effectively implemented in everyday practice.

Managing Injuries of the Neck Trial (MINT): design of a randomised controlled trial of treatments for whiplash associated disorders

BackgroundA substantial proportion of patients with whiplash injuries develop chronic symptoms. However, the best treatment of acute injuries to prevent long-term problems is uncertain. A stepped care treatment pathway has been proposed, in which patients are given advice and education at their initial visit to the emergency department (ED), followed by review at three weeks and physiotherapy for those with persisting symptoms. MINT is a two-stage randomised controlled trial to evaluate two components of such a pathway: 1. use of The Whiplash Book versus usual advice when patients first attend the emergency department; 2. referral to physiotherapy versus reinforcement of advice for patients with continuing symptoms at three weeks.MethodsEvaluation of the Whiplash Book versus usual advice uses a cluster randomised design in emergency departments of eight NHS Trusts. Eligible patients are identified by clinicians in participating emergency departments and are sent a study questionnaire within a week of their ED attendance. Three thousand participants will be included. Patients with persisting symptoms three weeks after their ED attendance are eligible to join an individually randomised study of physiotherapy versus reinforcement of the advice given in ED. Six hundred participants will be randomised. Follow-up is at 4, 8 and 12 months after their ED attendance. Primary outcome is the Neck Disability Index (NDI), and secondary outcomes include quality of life and time to return to work and normal activities. An economic evaluation is being carried out.ConclusionThis paper describes the protocol and operational aspects of a complex intervention trial based in NHS emergency and physiotherapy departments, evaluating two components of a stepped-care approach to the treatment of whiplash injuries. The trial uses two randomisations, with the first stage being cluster randomised and the second individually randomised.

How much does pre-trial testing influence complex intervention trials and would more testing make any difference? An email survey

BackgroundThe UK Medical Research Council has proposed that complex interventions should be tested in exploratory trials prior to a full-scale trial so as to better define the intervention and test the feasibility of components such as recruitment. It is not clear to what extent this is being done. This study aimed to determine to what extent complex interventions are tested prior to a full-scale trial and whether more or different testing would have led to a different intervention being used in the trial.MethodsEmail survey of the authors of complex intervention trials published in seven major journals in 2004.Results72% (50/69) of eligible authors replied. Eight authors did not consider their interventions to be complex. The majority of respondents' complex interventions were tested (34/42): some extensively. Conversely, only 17 of the 34 published reports describing these trials mention testing. Two-thirds (22/34) of those testing their interventions did not believe that more or different testing would have produced a more effective intervention. 31% (13/42) of all authors did believe further testing would have led to improvements. Five respondents mentioned a lack of funding as a reason for not doing more testing.ConclusionComplex interventions are generally tested prior to their evaluation in a full-scale trial, although the amount of testing varies. Testing is often not described in trial reports, which makes it hard to judge whether a trial result could be improved with a better intervention, or whether further work with a different intervention is required.

Retrofitting houses with insulation to reduce health inequalities: Aims and methods of a clustered, randomised community-based trial

This paper describes the purpose and methods of a single-blinded, clustered and randomised trial of the health impacts of insulating existing houses. The key research question was whether this intervention increased the indoor temperature and lowered the relative humidity, energy consumption and mould growth in the houses, as well as improved the health and well-being of the occupants and thereby lowered their utilisation of health care. Households in which at least one person had symptoms of respiratory disease were recruited from seven predominantly low-income communities in New Zealand. These households were then randomised within communities to receive retrofitted insulation either during or after the study. Measures at baseline (2001) and follow-up (2002) included subjective measures of health, comfort and well-being and objective measures of house condition, temperature, relative humidity, mould (speciation and mass), endotoxin, beta glucans, house dust mite allergens , general practitioner and hospital visits, and energy or fuel usage. All measurements referred to the three coldest winter months, June, July and August. From the 1352 households that were initially recruited, baseline information was obtained from 1310 households and 4413 people. At follow-up, 3312 people and 1110 households remained, an 84% household retention rate and a 75% individual retention rate. Final outcome results will be reported in a subsequent paper. The study showed that large trials of complex environmental interventions can be conducted in a robust manner with high participation rates. Critical success factors are effective community involvement and an intervention that is valued by the participants.

Guiding the design and selection of interventions to influence the implementation of evidence-based practice: an experimental simulation of a complex intervention trial

A consistent finding in health services research is the report of uneven uptake of research findings. Implementation trials have a variable record of success in effectively influencing clinicians’ behaviour. A more systematic approach may be to conduct Intervention Modelling Experiments before service-level trials, examining intervention effects on ‘interim endpoints’ representing clinical behaviour, derived from empirically supported psychological theories. The objectives were to: (1) Design Intervention Modelling Experiments by backward engineering a ‘real-world’ randomised controlled trial (NEXUS); (2) examine the applicability of psychological theories to clinical decision-making; (3) explore whether psychological theories can illuminate how interventions achieve their effects. A 2×2 factorial randomised controlled trial was designed with pre- and post-intervention data collection by postal questionnaire surveys. The first survey was used to generate feedback data and the interventions were delivered in the second survey. General medical practitioners (GPs) in England and Scotland participated. First survey respondents were randomised twice to receive or not audit and feedback and educational reminder messages. The main outcome measures included behavioural intention (general plan to refer for lumbar X-rays) and simulated behaviour (specific, scenario-based, decisions to refer for lumbar X-ray). Predictors were attitude, subjective norm, perceived behavioural control (theory of planned behaviour), self-efficacy (social cognitive theory) and decision difficulty. Both interventions significantly influenced simulated behaviour, but neither influenced behavioural intention. There were no interaction effects. All theoretically derived cognitions significantly predicted simulated behaviour. Only subjective norm was not predictive of behavioural intention. The effect of audit and feedback on simulated behaviour was mediated through perceived behavioural control. The results of this study suggest that Intervention Modelling Experiments, using psychological models to help isolate mediators of clinical decision-making, may be a means of developing more potent interventions, and selecting implementation interventions with a greater likelihood of success in a service-level randomised controlled trial.

A lesson learnt: the importance of modelling in randomized controlled trials for complex interventions in primary care

The Randomised Controlled Trial (RCT) is recognised as the 'gold standard' in quantitative research. However RCTs testing health care interventions can be difficult to design and implement. Health care interventions are often complex in themselves and are always applied in complex settings. Such interventions require a process of careful 'modelling' to maximize the chances of successful trials that will add to knowledge. To describe the terms 'complex' and 'modelling' as used in the setting of randomised controlled trials of complex interventions. To give a practical example of an RCT involving a complex intervention applied in a health care setting to illustrate how this might take place in practice. We describe an RCT designed and conducted by the authors. We then use our trial as an example to illustrate how complex interventions such as ours might benefit from modelling during the design of the intervention and the setting within which the intervention is to be tested. Our project was designed and tested before current guidance on complex interventions was published; our RCT was therefore not 'modelled' but was based on the outcome of a single quantitative pilot study. As part of our study we ran a parallel qualitative study, which highlighted several areas of complexity both in our intervention, and in the setting within which we applied it. In this paper we show how modelling might have allowed us to recognise these complexities at an early stage and might therefore have resulted in a study more likely to have demonstrated useful outcomes. Careful modelling of complex interventions is an essential step in designing trials of innovations in health care and health care services. Such a process ensures that interventions fit with and reflect the complexities of the settings within which interventions will be applied, and should ensure that the outcomes chosen are those most appropriate to demonstrate any benefits or risks.

Application of the development stages of a cluster randomized trial to a framework for valuating complex health interventions.

IntroductionTrials of complex health interventions often pose difficult methodologic challenges. The objective of this paper is to assess the extent to which the various development steps of a cluster randomized trial to optimize antibiotic use in nursing homes are represented in a recently published framework for the design and evaluation of complex health interventions. In so doing, the utility of the framework for health services researchers is evaluated.MethodsUsing the five phases of the framework (theoretical, identification of components of the intervention, definition of trial and intervention design, methodological issues for main trial, promoting effective implementation), corresponding stages in the development of the cluster randomized trial using diagnostic and treatment algorithms to optimize the use of antibiotics in nursing homes are identified and described.ResultsSynthesis of evidence needed to construct the algorithms, survey and qualitative research used to define components of the algorithms, a pilot study to assess the feasibility of delivering the algorithms, methodological issues in the main trial including choice of design, allocation concealment, outcomes, sample size calculation, and analysis are adequately represented using the stages of the framework.ConclusionsThe framework is a useful resource for researchers planning a randomized clinical trial of a complex intervention.