Implementation Of Routine Outcome Monitoring Research Articles

Routine outcome monitoring and feedback in alcohol and other drug treatment: A qualitative study of client perspectives on implementation.

Routine outcome monitoring (ROM) and feedback is an evidence-supported strategy for evaluating alcohol and other drug (AOD) treatment outcomes. However, the implementation of ROM and feedback into AOD services remains a significant challenge. Research aimed at understanding client perspectives on ROM and feedback is needed to facilitate successful implementation. This study examined experiences with and perceptions of ROM and feedback in a sample of clients receiving AOD treatment. Interviews and online surveys were conducted with N = 26 people (13 male; Mage = 36.12 years, SD = 10.29) enrolled in an AOD treatment program in Australia. Data analysis of the transcripts was guided by thematic analysis, while descriptive statistics were used to analyse quantitative survey data. Four major themes were identified in the qualitative data: (i) ROM and feedback is valuable to AOD treatment; (ii) clear and concise outcome measures with an integrated feedback loop are vital to reliable ROM; (iii) desire for visual and verbal feedback that highlights progress; and (iv) ROM and feedback can be emotionally challenging. Participants valued ROM when it was clearly integrated within AOD treatment and they received feedback on their responses. Potential facilitators to implementing and improving the provision of ROM and feedback in AOD treatment include: (i) a clear, treatment-based rationale to foster client buy-in for ROM and maximise AOD treatment benefit; (ii) brief outcome measure surveys; and (iii) graphical visualisations of ROM feedback.

Routine outcome monitoring: The need for case examples

Routine outcome monitoring (ROM) involves measuring an outcome variable (e.g., symptomsand alliance) at a regular interval over time (e.g., before each session), using a standardized patient self-report measure. Several computerized ROM system exists that also provide the therapist with benchmarks for these measures based on expected treatment responses (ETRs) for a patient, and clinical support tools (CSTs) to help the therapists in making clinician decisions in the treatment. When implementing ROM into clinical practice, a therapist needs to decide what patient-self report measure to use as well as what ROM system might be most clinically helpful and cost-effective. Although the use of ROM has shown to be effective, especially in reducing dropout and reducing deterioration of patients, the implementation of ROM can be challenging. The use of ROM always requires the active involvement of the therapist, but the implementation in treatment may look different for each patient. It is thus helpful to see case illustration of exactly how ROM might be useful or not so useful in clinical practice.

Applying behaviour change theory to understand the barriers to implementing routine outcome monitoring.

Routine outcome monitoring (ROM) is a valuable tool for monitoring client progress and pre-empting deterioration, however, there is considerable variation in how data are collected and recorded and uptake in clinical practice remains low. The aim of this study was to develop a self-report measure of practitioner attitudes to ROM in order to better understand the barriers to successful implementation in Child and Adolescent Mental Health Services (CAMHS). An anonymous survey was completed by 184 CAMHS practitioners in the United Kingdom. The survey was designed using the Capability, Opportunity, and Motivation Model of Behaviour (COM-B). Practitioners who reported using ROM frequently in their clinical work (53%) were compared to those who used ROM infrequently (47%) across dimensions of the COM-B survey subscales. Confirmatory factor analysis confirmed the proposed four-factor structure, showing acceptable model fit, with high factor loadings and good reliability for all subscales. Frequent users of ROM exhibited significantly higher psychological capability, physical opportunity, social opportunity, and motivation, compared to infrequent users F (4, 140) = 14.76, p < .0001; Pillai's Trace = .297, partial η2 = .30. Results highlight several barriers to ROM, including the belief that there is not a strong evidence base for ROM, not receiving external training, and not discussing feedback and outcome data in supervision. In the hope of improving the successful implementation of ROM, this research provides an evidence-based tool for assessing practitioners' attitudes to ROM, which map on to intervention functions and represent targets for future implementation efforts. The value of routine outcome monitoring (ROM) as a means to measure client progress and to elevate the efficiency and quality of mental health care is well-documented in the research literature, however, uptake in practice remains relatively low. This study applied behaviour change theory to develop a psychometrically sound self-report measure of practitioners' perspectives and practices to understand the barriers to implementation in child and adolescent mental health services in the United Kingdom. The complex and multifaceted nature of the barriers to implementation requires multilevel behaviour change strategies at the client, clinician, and organisational level. Recommendations for practice include the need for integrated, multilevel strategies aimed at improving practitioners' capabilities and motivations, strong organisational leadership and a culture of data gathering and sharing, and implementation interventions, which are tailored to target local barriers.

Challenges in Investigating the Effective Components of Feedback from Routine Outcome Monitoring (ROM) in Youth Mental Health Care

BackGroundStudies on feedback in youth mental health care are scarce and implementation of feedback into clinical practice is problematic.ObjectiveTo investigate potentially effective components of feedback from Routine Outcome Monitoring (ROM) in youth mental health care in the Netherlands through a three-arm, parallel-group, randomized controlled trial in which a literature-based, multi-faceted implementation strategy was used.MethodParticipants were randomly allocated to three conditions (basic feedback about symptoms and quality of life; basic feedback supplemented with clinical support tools; discussion of the feedback of the second condition with a colleague while following a standardized format for case consultation) using a block randomization procedure, stratified by location and participants’ age. The youth sample consisted of 225 participants (mean age = 15.08 years; 61.8% female) and the parent sample of 234 mothers and 54 fathers (mean age of children = 12.50 years; 47.2% female). Primary outcome was symptom severity. Secondary outcomes were quality of life and end-of-treatment variables. Additionally, we evaluated whether being Not On Track (NOT) moderated the association between condition and changes in symptom severity.ResultsNo significant differences between conditions and no moderating effect of being NOT were found. This outcome can probably be attributed to limited power and implementation difficulties, such as infrequent ROM, unknown levels of viewing and sharing of feedback, and clinicians’ poor adherence to feedback conditions.ConclusionsThe study contributes to our limited knowledge about feedback from ROM and underscores the complexity of research on and implementation of ROM within youth mental health care.Trial registration Dutch Trial Register NTR4234 .

Adventure Therapy and Routine Outcome Monitoring of Treatment: The Time Is Now

Background: Routine outcome monitoring (ROM) was popularized in the mid-1990s to improve client outcomes in psychotherapy, though implementation in clinical practice has been slow. Although increased outcome research in adventure therapy (AT) in the last decade has demonstrated AT as a viable treatment option, recent reviews have found worrying trends regarding research methodology and poorly substantiated claims of superiority. Purpose: The purpose of this article is to explore the potential for ROM in AT. Methodology/Approach: We conducted a brief review of the literature on ROM and offered a discussion that positions principles of ROM with the nascent knowledge base of AT. Findings/Conclusions: We propose ROM is a viable next step in AT research and practice. ROM can explore when change is likely to occur during an AT program and provide a platform for improving client engagement and outcomes. Implications: We recommend implementation of ROM in AT and that future AT research explore therapist effects and important therapeutic factors, such as the therapeutic alliance and deterioration.

A qualitative exploration of SMART Recovery meetings in Australia and the role of a digital platform to support routine outcome monitoring

Previous research has reported on the benefits of mutual support groups. However, such groups do not routinely collect data on participant outcomes. Moreover, the effect of collecting outcomes measures on these groups is unknown. The objective of this mixed methods study was to elicit participant views on using a novel, purpose built digital platform for routine outcome monitoring (ROM) as a standard component of a mutual support group. SMART Recovery, or the Self-Management and Recovery Training program, is group-based and uses professional clinicians to facilitate discussion and foster mutual support for a range of addictive behaviours, alongside Cognitive Behavioural Therapy and Motivational Interviewing techniques. This paper reports on the qualitative component of this study and how participants perceive ROMs, and the potential shift to technological resources. Twenty semi-structured telephone interviews were conducted with participants from SMART Recovery groups across New South Wales, Australia. Participants discussed their use of mutual support within group meetings to manage their recovery, including: naming their goals in front of peers; learning from clinicians and group discussion; and developing reciprocal and caring relationships. They also described any previous experience with routine outcomes measures and how digital technologies might enhance or hinder group function. Participants valued mutual support groups and reported that digital technologies could be complementary to physical, weekly group meetings. They were also concerned that the introduction of technological resources might pose a threat to physical meetings, thereby risking their access to mutual support. Findings have implications for the implementation of ROM when delivered via digital mechanisms, and indicate threats and opportunities that warrant consideration for future initiatives.

Attitudes Towards the Use of Routine Outcome Monitoring of Psychological Therapies Among Mental Health Providers: The EBPAS\u2013ROM

Implementation of routine outcome monitoring (ROM) in mental health care is progressing slowly. Knowledge about factors influencing ROM implementation, including health providers’ attitudes towards ROM, is necessary. Based on a survey of 662 psychologists and nurses, this article describes (1) the development of a short instrument measuring provider attitudes towards ROM, derived from the Evidence-based Practice Attitude Scale (EBPAS), and (2) how attitudinal domains relate to clinicians’ current use of standardized instruments for treatment evaluation. The EBPAS–ROM showed concurrent validity in predicting aspects important for the implementation of ROM, including perceived limitations and the value of organizational support.

Provider-Associated Measurement Error in Routine Outcome Monitoring in Community Mental Health.

Despite the strengths of routine outcome monitoring (ROM) in community mental health settings, there are a number of barriers to effective implementation of ROM, including measurement error due to provider factors (e.g., training level) and non-target client factors (i.e., client characteristics which have no meaningful relationship to the outcome of interest). In this study, ROM data from 80 client-provider dyads were examined for sources of variance due to provider factors and non-target client factors. Results indicated that provider factors and non-target client factors accounted for between 9.6 and 54% of the variance in the ROM measures. Our findings supported past research that provider characteristics impact ROM, and added the novel finding that client gender, age, diagnosis, and cognition also impact ROM. Methods to increase accuracy and utility of ROM in community mental health are discussed.

Implementing routine outcome monitoring in public mental health services in Israel: Shared and unique challenges.

This study aimed to assess the feasibility, benefits, and challenges of routine outcome monitoring (ROM) in a public mental health centre in Israel. This is the first initiation of ROM implementation in a setting of a large psychiatric hospital, affiliated with the largest health maintenance organization in Israel, Clalit Health Services. Four ambulatory units were invited to participate in the implementation of ROM. Prior to initiation of recruitment, staff meetings were held and broad descriptions of current knowledge regarding benefits and challenges were discussed. Upon initiation of recruitment, patients completed outcome and process questionnaires during each session, and reports of patients' progress were sent to therapists soon after sessions ended. One hundred sixty-three patients were invited to participate in measurement and feedback of therapy outcomes. Overall, 101 patients and 32 therapists agreed and actively participated in the implementation phase of ROM, producing 535 evaluated measurements and reports. Clinical, methodological, legal, administrative, and ethical aspects of the implementation were encountered and documented throughout the implementation process. Medical confidentiality issues, as well as the multidisciplinary nature of hospital staff work, serve as central and unique challenges for incorporating ROM in a public psychiatric hospital. Other challenges that were previously reported in other implementation studies were also encountered and included therapist overload and objections, attrition, and the need for organizational support. Recommendations for future pioneering efforts for ROM implementation in large psychiatric facilities are discussed.

System-Wide Implementation of Routine Outcome Monitoring and Measurement Feedback System in a National Network of Operational Stress Injury Clinics.

This manuscript reviews the development and implementation process of the Client Reported Outcome Monitoring Information System in the VA Canada Operational Stress Injury National Network, and reports on outcomes of an evaluation to assess usage, barriers, and facilitators to implementation. The majority of clinicians reported regular use of routine outcomes monitoring, although objective data suggested somewhat lower actual use. In general, clinicians endorsed all barriers and most facilitators as influencing their use of routine outcomes monitoring in a minor way. However, users and non-users differed in their endorsement of facilitators and barriers. Implications for research and implementation efforts are discussed.

Health Information Technologies-Academic and Commercial Evaluation (HIT-ACE) methodology: description and application to clinical feedback systems.

BackgroundHealth information technologies (HIT) have become nearly ubiquitous in the contemporary healthcare landscape, but information about HIT development, functionality, and implementation readiness is frequently siloed. Theory-driven methods of compiling, evaluating, and integrating information from the academic and commercial sectors are necessary to guide stakeholder decision-making surrounding HIT adoption and to develop pragmatic HIT research agendas. This article presents the Health Information Technologies—Academic and Commercial Evaluation (HIT-ACE) methodology, a structured, theory-driven method for compiling and evaluating information from multiple sectors. As an example demonstration of the methodology, we apply HIT-ACE to mental and behavioral health measurement feedback systems (MFS). MFS are a specific class of HIT that support the implementation of routine outcome monitoring, an evidence-based practice.ResultsHIT-ACE is guided by theories and frameworks related to user-centered design and implementation science. The methodology involves four phases: (1) coding academic and commercial materials, (2) developer/purveyor interviews, (3) linking putative implementation mechanisms to hit capabilities, and (4) experimental testing of capabilities and mechanisms. In the current demonstration, phase 1 included a systematic process to identify MFS in mental and behavioral health using academic literature and commercial websites. Using user-centered design, implementation science, and feedback frameworks, the HIT-ACE coding system was developed, piloted, and used to review each identified system for the presence of 38 capabilities and 18 additional characteristics via a consensus coding process. Bibliometic data were also collected to examine the representation of the systems in the scientific literature. As an example, results are presented for the application of HIT-ACE phase 1 to MFS wherein 49 separate MFS were identified, reflecting a diverse array of characteristics and capabilities.ConclusionsPreliminary findings demonstrate the utility of HIT-ACE to represent the scope and diversity of a given class of HIT beyond what can be identified in the academic literature. Phase 2 data collection is expected to confirm and expand the information presented and phases 3 and 4 will provide more nuanced information about the impact of specific HIT capabilities. In all, HIT-ACE is expected to support adoption decisions and additional HIT development and implementation research.

Routine outcome monitoring and process quality in mental health care: a descriptive study in daily practice.

Routine outcome monitoring (ROM) is used to improve quality of health care by improving the process of care. ROM was introduced nationwide in mental health care in the Netherlands with the primary goal of enhancing the quality of treatment. This study aims to establish the impact of ROM on the process quality of treatment in the daily practice of secondary mental health care in the Netherlands. Two cohorts from the same treatment programme were identified, before and after implementation of ROM. Data on patient characteristics, presence of baseline ROM measurement and indicators of the quality of the treatment process were retrieved from the electronic health care information system. Patient data were collected until 6 months after enrolment in the treatment programme. The magnitude of the effect of ROM on the quality of the treatment process was established. The pre-ROM cohort comprised 271 patients and 231 patients constituted the post-ROM cohort. In the ROM cohort, both indicators for process quality 'presence of treatment plan' and 'presence of cancellations' were significantly higher; the odd ratios were 3,37 [confidence interval (CI) 2.32-4.89] and 1,63 (CI 1.14-2.33), respectively. Indicators on process quality are important for understanding and establishing the impact of ROM in daily practice of secondary mental health care. Implementing ROM moderately increased the presence of a treatment plan. While this suggests improved treatment agreement, it did not result in better compliance of patients with individual treatment activities, as presence of cancellations was significantly higher after introduction of ROM.

Attitudes of Patients and Clinicians Towards Routine Outcome Monitoring in Clinical Practice.

Background The aim of Routine Outcome Monitoring (ROM) is to improve the quality of clinical treatment. Providing feedback on the outcome of treatment to both clinicians and patients is an integral part of ROM, allowing modifications to treatment. Published studies are inconsistent with respect to patient attitudes towards ROM. Clinicians, however, generally appear to be more negative than enthusiastic. Although Routine Outcome Monitoring is currently implemented throughout the Netherlands, the process is not going as smoothly as first envisaged. Research into patient and clinician attitudes towards ROM can offer a partial explanation of the difficulties involved. Aim The aim of this study is to gain insight into the attitudes of both patients and clinicians towards ROM and the factors that play a role in the implementation of ROM in a general, non-academic psychiatric outpatient setting. Method A study was carried out into the attitudes of both patients and clinicians towards the recently introduced ROM procedure. Specifically designed attitude rating scales were presented to a group of 176 patients and their 69 clinicians in the course of their treatment. Outcomes and conclusions Forty-six percent of the patients consulted and 39% of their clinicians completed an attitude rating scale. Patients were generally positive about the use of ROM during the clinical care process. They found it useful and did not find it stressful. The use of a computer during this process was considered to be a positive facet. Clinicians, on the other hand, had a negative reaction and rejected ROM. In the discussion possible explanations for the cause of these findings are offered.

Embedding Routine Outcome Monitoring in the Operations Management of a Care Pathway for Depression and Anxiety. Case Illustration and Lessons Learned in the Netherlands.

Background Routine Outcome Monitoring (ROM) is a new phenomenon in the mental healthcare system. Its introduction into daily clinical practice has not been straightforward. This study examines practical examples of the embedding of this innovation and its effective acceptance in clinical practice. Method A case history illustrates how ROM was implemented in a large mental health hospital in The Netherlands. During implementation a number of weak points in the implementation process emerged. An intense analysis of the organization of the care process, in the context of care pathways and operations management, offered explanations of what went wrong in this case. Future improvements are suggested from an operations management viewpoint. Results and conclusion For the improved implementation of ROM in the mental healthcare sector implementation must be widely participatory, the process must be continuously monitored, and a leading, well-informed clinician must be appointed as process manager to guide the implementation process.

Session-by-session outcome monitoring in CAMHS: clinicians’ beliefs

AbstractThe Children and Young People's Improving Access to Psychological Therapies (CYP IAPT) programme emphasizes the meaningful contribution session-by-session routine outcome monitoring (ROM) can make to clinical practice and its importance in highlighting services’ effectiveness. Two studies on issues related to the implementation of ROM in children's services were conducted. Study 1 was qualitative; 12 Child and Adolescent Mental Health Services (CAMHS) professionals participated in focus groups. Themes identified included the idea that ROM could provide objectivity, could be collaborative and empowering. Concerns included how measures may adversely influence therapeutic sessions and how the information may be used by the service. These themes were used to develop a questionnaire about professionals’ experience of and views on session-by-session ROM. In Study 2, 59 professionals from four CAMHS teams completed the questionnaire. It was found that only 6.8% reported ‘almost always’ utilizing session-by-session ROM. Detailed analysis of questionnaire responses suggested two subscales reflecting the perceived negative and positive impact of session-by-session ROM. It was found that clinicians who currently use session-by-session ROM hold stronger positive and negative beliefs than clinicians who do not. This study suggests that session-by-session ROM is not currently routine practice within CAMHS and highlights the importance of considering how this practice can be best implemented within this setting with reference to clinician attitudes.

Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions

This article reviews the benefits, obstacles, and challenges that can hinder (and have hindered) implementation of routine outcome monitoring in clinical practice. Recommendations for future routine outcome assessment efforts are also provided. Spanning three generations, as well as multiple developed tools and approaches, the four authors of this article have spent much of their careers working to address these issues and attempt to consolidate this learning and experience briefly here. Potential “elephants in the room” are brought into the discussion wherever relevant, rather than leaving them to obstruct silently the field's efforts. Some of these topics have been largely ignored, yet must be addressed if we are to fulfill our promise of integrating science and practice. This article is an attempt to identify these important issues and start an honest and open dialogue.

Clinicians’ Perspectives on a Web-Based System for Routine Outcome Monitoring in Old-Age Psychiatry in the Netherlands

BackgroundIn health care, the use of physical parameters to monitor physical disease progress is common. In mental health care, the periodic measurement of a client’s functioning during treatment, or routine outcome monitoring, has recently become important. Online delivery of questionnaires has the potential to reduce clinicians’ resistance to the implementation of routine outcome monitoring. Online delivery enables clinicians to receive results on a questionnaire in a graphic directly after data entry. This gives them insight into the progress of a client at a single glance.ObjectiveTo explore clinicians’ perspectives on a routine outcome monitoring procedure where questionnaires and feedback on scores were delivered online. Questionnaires could also be filled out on paper and then entered into the online system by a research assistant.MethodsIn 2009 we sent an online survey, consisting of five yes-or-no questions and six open-ended questions, to all clinicians in the 14 mental health care organizations working with the routine outcome monitoring system in the Netherlands. Of the 172 clinicians contacted, 80 (47%) opened the link and 70 of these 80 (88%) clinicians completed the survey.ResultsClinicians seldom used the graphical feedback from the Web-based system, which indicates that direct feedback on scores did not enhance the implementation of routine outcome monitoring. Integration into the electronic patient record and more training on interpretation and implementation of feedback in daily practice were seen as the primary points for further improvement. It was mainly the availability of a research assistant that made the routine outcome monitoring procedure feasible.ConclusionsWithout a research assistant and training in the interpretation of outcomes, software programs alone cannot ensure effective implementation of monitoring activities in everyday practice.

Routine outcome monitoring in the Netherlands: practical experiences with a web‐based strategy for the assessment of treatment outcome in clinical practice

Routine outcome monitoring (ROM) is a method devised to systematically collect data on the effectiveness of treatments in everyday clinical practice. ROM involves documenting the outcome of treatments through repeated assessments. Assistants are employed who perform a baseline assessment comprising a standardized diagnostic interview, administration of rating scales and completion of several self-report measures by the patient. At fixed time intervals, assessments are repeated. Dedicated Web-based software has been developed to assist in this task. ROM informs therapists and patients on the severity of the complaints at intake, and the waxing and waning of symptoms over the course of treatment. Researchers can use ROM for effectiveness research, and managers can use it for benchmarking. The use of ROM for research is illustrated by presenting data on the diagnostic status of patients participating in ROM and data on treatment outcome of a subgroup of patients (with panic disorder) in our database. The results show that implementation of ROM is feasible, and after some initial reservations, most therapists now consider ROM to be a necessary and important adjunct to the clinical treatment. In addition, ROM furthers research as the data can be used to study the phenomenology of psychiatric disorders and the outcome of treatments delivered in everyday practice.