Fidelity Reviews Research Articles

A demonstration project involving peers as providers of evidence-based, supported employment services.

The present demonstration project involved development of a training program designed to teach recovering consumers employed as peer advocates how to provide evidence-based supported employment services to consumers with severe mental illness. A training curriculum was developed to teach the core competencies of the Individual Placement and Support (IPS) model of supported employment. Three peers participated in training and provided work outcome data from their caseloads. Assessments were conducted of peers' competence in implementing IPS and effectiveness in promoting job placements. Peer competency was assessed by the following: (a) a formal IPS fidelity review performed by two external reviewers to evaluate service implementation, and (b) the Kansas Employment Specialist Job Performance Evaluation, an objective measure of employment specialist attitudes and skills. Program efficacy was assessed by examining the number of job placements and corresponding tenure. The fidelity review revealed that peers met IPS standards of implementation on 7 of 14 items assessing service delivery. The Kansas scale results revealed attitudes to be a relative strength and job performance competency ratings fell in the average to above average range across skill areas assessed (e.g., vocational assessment, job development). Thirty-three percent of consumers from the peers' caseloads got competitive jobs; mean tenure was 26.1 weeks. This demonstration project provides a starting point for future efforts aimed at expanding the role of peers as providers of evidence-based mental health services and provides a measured degree of optimism that this is a realistic, attainable goal.

Abstract 2075: Combination tTerapy KPT-SINE (selective inhibitors of nuclear export) with radiotherapy have additive effects in non-small cell lung cancer (NSCLC) cells in vitro and in vivo.

Abstract Radiation therapy is an established therapeutic option for lung cancer, but its use is limited due to side effects. Combining radiosensitizers with RT is a way to augment the efficacy of RT. Chromosome Region Maintenance Protein 1/Exportin 1 (CRM1/XPO1) is a key nuclear export protein, the cargoes of which are Tumor Suppressor Proteins (TSPs) such as p53, FOXO, PTEN, pRB and I-κB. Small molecule SINE block CRM1-dependent nuclear export and force the nuclear retention of TSPs, potentially sensitizing cells to RT. Suggested mechanisms include: 1) Induction of “genome fidelity review” by forced nuclear retention of TSPs; 2) Prevention of single stranded DNA break repair, and thereby reduction of resistance of cancer cells to RT. Methods: IC50 of KPT-SINE e.g. KPT-185 and clinical compound KPT-330, were determined in MTT assays using human NSCLC p53 wild type (wt) cancer cell lines H1299 and A549. Clonogenic assays in the SINE sensitive H1299 (IC50 100nM) and SINE-resistant A549 (IC50 1.7μM) lines were conducted in the presence or absence of escalating doses of radiation. An in vivo tumor xenograft model using the A-549 cell line in SCID mice was also conducted with KPT-330. Results: KPT-185 demonstrated inhibitory effects on the clonogenicity of two NSCLC cell lines in a dose dependent fashion. We showed that, while RT alone treatment caused coloines growing curve reduced to less than <40% of colonies in both cell lines. Combination of both agents (RT and SINE at IC25 concentrations) demonstrated an additive effect for A549 cell line, and a dose-related effect for H1299 cell line. In vivo, we demonstrated significant reductions in tumor growth, where treatment with KPT-330 caused a >75% tumor volume reduction when compared to vehicle, and a >65% reduction when compared to Cisplatin. In this model, KPT-330 was fairly well-tolerated, with minimal body weight loss by animals over the course of the study. Conclusions: These studies showed that KPT-SINE significantly inhibited the growth of two p53 wt NSCLC lines by MTT assay. Similar inhibition was observed in clonogenic assays with cells (A549) resistant to SINE. We described the additive effects of SINE and RT on NSCLC cell lines, suggesting that SINEs can act as radiosensitizers. We further demonstrated that SINEs are efficacious in vivo in the A549 xenograft model, and display superior efficacy when compared to Cisplatin. SINE compounds represent a promising novel therapy against NSCLC both as stand-alone agents and in combination with RT. These data support further development of SINE-based therapies for NSCLC. Further work on in vivo studies of KPT-SINE in combination with RT will be reported. Citation Format: Tami Rashal, Dilara McCauley, Maya Ilouze, Nir Raphael, Inessa Solomonik, Yaccov Lawrence, Ronen Shavit, Sharon Shacham, Michael Kauffman, Nir Peled. Combination tTerapy KPT-SINE (selective inhibitors of nuclear export) with radiotherapy have additive effects in non-small cell lung cancer (NSCLC) cells in vitro and in vivo. [abstract]. In: Proceedings of the 104th Annual Meeting of the American Association for Cancer Research; 2013 Apr 6-10; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2013;73(8 Suppl):Abstract nr 2075. doi:10.1158/1538-7445.AM2013-2075

Strengths Model Case Management Fidelity Scores and Client Outcomes

The study examined the relationship between fidelity of strengths model case management (SMCM) and client outcomes of psychiatric hospitalization, competitive employment, postsecondary education, and independent living. Data were collected over an 18-month period during regularly scheduled fidelity reviews for 14 case management teams representing ten agencies serving an average of 953 clients diagnosed as having a serious mental illness. Time-varying covariate linear growth modeling examined the relationship between fidelity scores and client outcomes. A statistically significant association was found between fidelity scores and psychiatric hospitalization, competitive employment, and postsecondary education. The study results offer promising evidence that higher SMCM fidelity has a positive effect on clients over an 18-month period, thereby providing an effective complement to current mental health treatment.

An Overview of Evidence-Based Mental Health

Can J Psychiatry. 2011;56(9):511-513. The preceding articles by Dr Rob Whitley and colleagues' and by Dr Eric A Latimer and colleagues2 review 2 critical aspects of evidence-based mental health (EBMH): incorporating recovery ideology and cultural diversity into research-supported practices. Striving for more effective mental health care will entail addressing these and other challenges. As the US Institute of Medicine3 described the situation, gap between what can and should be and what exists is so large that ... it constitutes a chasm.p 30 In this editorial, we identify 4 other contextual aspects of offering EBMH: practice implementation, decision making, program management, and systems oversight. For each, we describe the issue, the current state of mental health services, and possible future directions. Evidence-Based Practice Implementation Clinicians and program leaders want to support the recovery of their clients by offering practices that consistently promote the outcomes their clients seek. However, even for motivated implementers, the process of establishing a new evidence-based service is complex.4 Important facets of implementation outcome include practice acceptability, appropriateness, fidelity, affordability, and penetration.5 Current approaches to implementation encompass a combination of manuals, fidelity reviews, learning collaboratives, and regional technical assistance centres.6 Technical assistance centres are at the heart of efforts in most regions. Consultants in these centres provide local programs with administrative guidelines and advice, training of new clinicians, ongoing supervision by telephone, and regular visits to monitor progress and suggest further strategies.7,8 This approach has been moderately successful9 and has qualitative research support.7,10,11 However, the science of implementation is very young - the field needs rigorous evidence regarding implementation strategies.12 Emerging strategies for getting evidence-based practices (EBPs) to those who need them emphasize information technology, such as distance learning, telemedicine, computerized self-treatment, and electronic decision supports.13 In theory, electronic systems can incorporate evidence-based guidelines, individual tailoring (now called personalized medicine), and training for clinicians while they are providing services.14,15 Further, electronic records permit direct measurement of aspects of implementation, such as screening and practice penetration (the per cent of people who get a needed screening or service). Evidence-Based Decision Making High-quality health care decisions integrate patient preference, clinical judgment, and the constantly expanding corpus of scientific knowledge.3 Nevertheless, today most decisions rely on clinician judgment alone. Few practice sites routinely ask patients about their preferences.16 The scientific knowledge base is accessible, but using it requires a systematic approach to framing the clinical question, finding the current research, evaluating the quality of the research, deciding if the research applies to the current patient and situation, and using the research in a process of shared decision making.17 Mental health practitioners can master the techniques of evidence-based decision making,'8 but several common barriers reduce their use in everyday practice; for example, time constraints and lack of computer access. The fundamental challenge is to get patient preference, health status (symptoms, function, and side effects), and the relevant scientific evidence to the patient and the clinician in understandable formats at the time when a decision needs to be made.'9 To have direct impact on routine care, the clinician and the patient would need to be able to enter the patient's personal data, specify the question, and access the relevant evidence immediately. This kind of access to patient-relevant, up-to-date science is impossible now because current methods for asking and answering the questions are too complex and time-consuming for most real-world settings. …

Developing and validating measures for child welfare agencies to self-monitor fidelity to a child safety intervention

Building evidence of effective practice in child welfare requires practitioners and researchers to know the extent to which programs are implemented in order to understand evaluation results. Fidelity monitoring is a critical strategy for ensuring that evidence-based and promising practices are implemented as intended and can be studied in real-world contexts. This paper addresses challenges to measuring fidelity in child welfare systems and presents an approach taken with one state to define fidelity criteria and measure fidelity to a child safety intervention. Measurement challenges were addressed by using existing documents and case review mechanisms to assess fidelity, and measuring the quality of practitioner judgment using expert reviewers. Validity of fidelity criteria and fidelity review instruments was established through consensus with model developers and local purveyors. Twelve cases were reviewed by a panel of raters to assess inter-rater reliability and discrepancy between local purveyors and model developers. This participatory and capacity-building method can be replicated and used to develop and embed valid and reliable fidelity monitoring systems in public child welfare to continue to build evidence about what works in child welfare services.

A large‐scale dissemination and implementation model for evidence‐based treatment and continuing care.

Multiple evidence-based treatments for adolescents with substance use disorders are available; however, the diffusion of these treatments in practice remains minimal. A dissemination and implementation model incorporating research-based training components for simultaneous implementation across 33 dispersed sites and over 200 clinical staff is described. Key elements for the diffusion of the Adolescent Community Reinforcement Approach and Assertive Continuing Care were: (a) three years of funding to support local implementation; (b) comprehensive training, including a 3.5 day workshop, bi-weekly coaching calls, and ongoing performance feedback facilitated by a web tool; (c) a clinician certification process; (d) a supervisor certification process to promote long-term sustainability; and (e) random fidelity reviews after certification. Process data are summarized for 167 clinicians and 64 supervisors.

Individualization and Quality Improvement: Two New Scales to Complement Measurement of Program Fidelity

Fidelity scales have been widely used to assess program adherence to the principles of an evidence-based practice, but they do not measure important aspects of quality of care. Pragmatic scales measuring clinical quality of services are needed to complement fidelity scales measuring structural aspects of program implementation. As part of the instrumentation developed for the National Implementing Evidence-Based Practices Project, we piloted a new instrument with two 5-item quality scales, Individualization (a client-level quality scale) and Quality Improvement (an organizational-level quality scale). Pairs of independent fidelity assessors conducted fidelity reviews in 49 sites in 8 states at baseline and at four subsequent 6-month intervals over a 2-year follow-up period. The assessors followed a standardized protocol to administer these quality scales during daylong site visits; during these same visits they assessed programs on fidelity to the evidence-based practice that the site was seeking to implement. Assessors achieved acceptable interrater reliability for both Individualization and Quality Improvement. Principal components factor analysis confirmed the 2-scale structure. The two scales were modestly correlated with each other and with the evidence-based practice fidelity scales. Over the first year, Individualization and Quality Improvement improved, but showed little or no improvement during the last year of follow-up. The two newly developed scales showed adequate psychometric properties in this preliminary study, but further research is needed to assess their validity and utility in routine clinical practice.

Strategies for Improving Fidelity in the National Evidence-Based Practices Project

Background: The National Evidence-Based Practices (EBPs) Project developed and tested a model for facilitating the implementation of five psychosocial EBPs for adults with severe mental illness in the United States. Methods: The implementation model was tested in 53 sites in 8 states. In each site, one of the five EBPs was adopted for implementation and then studied for a 2-year period using a combination of qualitative and quantitative methods. Findings: At baseline, none of the sites had programs attaining high fidelity. Four factors were identified as influencing fidelity: (a) EBP-specific factors, (b) governmental factors, (c) leadership factors, and (d) fidelity review factors. Conclusion: A multipronged implementation strategy was effective in achieving high fidelity in over half of the sites seeking to implement a new EBP.

State Mental Health Policy: Washington State's Initiative to Disseminate and Implement High-Fidelity ACT Teams

State Mental Health Policy: Washington State's Initiative to Disseminate and Implement High-Fidelity ACT Teams

State Mental Health Policy: Washington State's Initiative to Disseminate and Implement High-Fidelity ACT Teams

This column describes Washington State's historic statewide initiative to implement ten high-fidelity assertive community treatment teams. Legislative support and administrative leadership facilitated the implementation of this evidence-based practice as part of a comprehensive transformation initiative within public mental health services for people with serious mental illness. Stakeholder feedback and initial fidelity reviews suggest that the first year has been successful. Crucial strategies that were important to the teams' successful implementation, such as training and consultation, are examined. Challenges, such as staff turnover, are described along with targeted approaches that have been integral to overcoming them. Next steps to promote sustainability and implications for state mental health authorities are discussed.

Critical strategies for implementing supported employment

Objective: This paper provides a review of experiences by six states and the District of Columbia in disseminating an evidence-based practice, supported employment (SE). Each jurisdiction had unique strengths and barriers to overcome to effectively implement supported employment services for people with severe mental illness. Methods: Using a case study approach seven jurisdictions report special aspects surrounding the implementation of evidence-based supported employment. Results: Diverse strategies were used to augment implementation of supported employment services: a) Instituting state-level administrative procedures and reconfiguration of local staffing to enhance collaboration between mental health and vocational rehabilitation; b) Promoting SE services through the media, on-line training, and training by early adopters; c) Hiring benefits specialists; d) Teaching outcome-based supervision; and e) Building capacity for supported employment fidelity reviews. Conclusions: Dissemination of evidenced-based supported employment was enhanced when six states and the District of Columbia addressed special aspects. Supported employment implementation included different pathways to good employment outcomes.