Mental Health Program Evaluation Research Articles

Using participant-oriented research in post-secondary mental health program development and evaluation

Purpose With the recent release of the National Standard for Mental Health and Well-Being for Post-Secondary Students, there is increased interest to integrate research and practice for mental health services on post-secondary campuses. Participant-oriented research is a useful framework to bridge this gap. This paper aims to describe the program development and evaluation process and reports challenges and lessons learned to inform future implementation strategies for similar endeavours. Design/methodology/approach A participant-oriented research approach was used to revise and evaluate an innovative interdisciplinary resilience program, entitled Roots of Resiliency, for post-secondary students. Findings This case analysis used the development and evaluation of Roots of Resiliency to demonstrate some of the strategies and challenges that exist for participant-oriented research related to mental health in the post-secondary context. Collaborative relationships among the various development team members contributed to an overall positive experience. Some challenges that others who work in post-secondary mental health field may consider include the need for content expertise, the ongoing need for communication among team members and the need for an effective system to give voice to all participants. Originality/value Any mental health program has a cultural component and is best co-developed by the particular students (e.g. indigenous students) who are to be served by the program. In this regard, the co-design and shared development and evaluation of the current mental health program is an example that can be emulated in other programs within the post-secondary context.

White House nominates Delphin‐Rittmon, for SAMHSA head

Miriam E. Delphin‐Rittmon, M.D., since 2015 the commissioner of the Connecticut State Department of Mental Health and Addiction Services, was nominated to be assistant secretary for mental health and substance use at the federal Department of Health and Human Services by the White House last week. In her role as SSA (Single State Authority), Delphin‐Rittmon has been committed to promoting recovery‐oriented, integrated and culturally responsive services and systems that foster dignity, respect and meaningful community inclusion. In addition, through her 20‐year career in the behavioral health field, Delphin‐Rittmon has extensive experience in the design, evaluation and administration of mental health, substance use and prevention services and programs, and has received several awards for advancing policy in these areas. Most recently, she received the 2019 State Service Award from the National Association of State Drug and Alcohol Directors and the 2016 Mental Health Award for Excellence from the United Nations Committee on Mental Health.

Problems faced by Syrian refugees and asylum seekers in Switzerland

BACKGROUND Refugees and asylum seekers are susceptible to developing common mental disorders due to their exposure to stressful experiences before, during and after their flight. The Syrian Civil War, which started in 2011, has led to a massive number of Syrians seeking refuge and asylum in European countries, including Switzerland. Currently, Syrians are the second-largest refugee and asylum-seeking population in Switzerland. However, very little is known about the problems faced by this new population in Switzerland and their needs relating to mental health services. Identifying the problems faced by this community is crucial to providing adapted and tailored mental health services to Syrian refugees in Switzerland. AIM OF THE STUDY The current study aimed to identify problems that Syrian refugees and asylum seekers face daily while living in Switzerland in order to inform the adaptation of a brief psychological intervention. METHODS We used a cross-sectional, qualitative design and collected data according to The Manual for Design, Implementation, Monitoring and Evaluation of Mental Health and Psychosocial Assistance Programs for Trauma Survivors in order to identify problems perceived by the target population. Free-listing, open-ended interviews were conducted with 30 adult Syrian refugees and asylum seekers and analyzed using thematic analysis. RESULTS The results show that besides physical health problems, Syrians experience primarily two types of problems: practical and psychological (emotional) problems. These two types of problems are closely interrelated. The most common practical problems (problems with government and authorities, problems related to residence permits, problems with integration, cultural differences, language problems, problems related to education, problems related to employment, and problems with housing) were reported by almost half of all participants. Symptoms of mental disorders and feelings of uncertainty, frustration and injustice were the most common psychological problems and were mentioned by more than one third of the participants. The finding that almost half of the participants reported typical symptoms of mental health disorders suggests that a considerable number of Syrian refugees and asylum seekers might need mental healthcare. CONCLUSIONS Authorities, practitioners and researchers should recognize that Syrian refugees and asylum seekers are strongly affected by a broad range of problems. Besides practical problems, they suffer a multitude of psychological problems, and a significant number of them report, among other issues, symptoms of mental health disorders. Officials working with this population should be aware of this vulnerability and be prepared to refer clients in need of mental healthcare to mental healthcare providers. Moreover, the significant variety and number of problems experienced by this population should be taken into consideration when developing solutions tailored to their needs.

When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory.

There is a need for accurate and efficient assessment tools that cover a range of mental health and psychosocial problems. Existing, lengthy self-report assessments may reduce accuracy due to respondent fatigue. Using data from a sample of adults enrolled in a psychotherapy randomized trial in Thailand and a cross-sectional sample of adolescents in Zambia, we leveraged Item Response Theory (IRT) methods to create brief, psychometrically sound, mental health measures. We used graded-response models to refine scales by identifying and removing poor performing items that were not well correlated with the underlying trait, and by identifying well-performing items at varying levels of a latent trait to assist in screening or monitoring purposes. In Thailand, the original 17-item depression scale was shortened to seven items and the 30-item Posttraumatic Stress Scale (PTS) was shortened to 10. In Zambia, the Child Posttraumatic Stress Scale (CPSS) was shortened from 17 items to six. Shortened scales in both settings retained the strength of their psychometric properties. When examining longitudinal intervention effects in Thailand, effect sizes were comparable in magnitude for the shortened and standard versions. Using Item Response Theory (IRT) we created shortened valid measures that can be used to help guide clinical decisions and function as longitudinal research tools. The results of this analysis demonstrate the reliability and validity of shortened scales in each of the two settings and an approach that can be generalized more broadly to help improve screening, monitoring, and evaluation of mental health and psychosocial programs globally.

Profiles of Quality of Life in a Homeless Population.

Quality of life (QOL) is a key indicator in mental health planning, program evaluation, and evaluation of patient outcomes. Yet few studies have focused on QOL in homeless populations. More specifically, research has yet to identify profiles of homeless individuals based on their QOL using cluster analysis. This study developed a typology of QOL for a sample of 455 homeless individuals recruited from 27 community and public organizations in Quebec (Canada). The typology was developed based on QOL scores, as well as sociodemographic, clinical, and service use variables. Study participants had to be at least 18 years old, with current or previous experience of homelessness. A questionnaire including socio-demographics, residential history, service utilization, and health-related variables was administered. Four clusters were identified using a two-step cluster analysis. QOL was highest in the cluster consisting of older women with low functional disability, and relatively few episodes of homelessness. The second cluster with high QOL scores included individuals living in temporary housing with relatively few mental health or substance use disorders (SUDs). The third cluster with low QOL included middle-aged women living in temporary housing, with criminal records, personality disorders, and SUDs. QOL was also lower in the fourth cluster composed of individuals with multiple homeless episodes and complex health problems as well as high overall service use. Findings reinforced the importance of disseminating specific programs adapted to the diverse profiles of homeless individuals, with a view toward increasing their QOL.

Evaluation of students' mental and social health promotion educational programs: A systematic review

INTRODUCTION:Various programs are implemented internationally to promote the mental and social health of the students in schools. This study systematically reviewed and categorized all resources, indicators, and criteria for evaluating mental and social programs of schools.MATERIALS AND METHODS:This systematic review was conducted by collecting data from the PubMed, Google Scholar, Scopus, ProQuest, and Web of Science databases using the keywords of “evaluation, mental health program, social health program, behavioral and emotional program.” In the initial review, 4295 studies were found, which reduced to 75 after removing the repetitions and evaluating the studies’ quality. The articles were selected using the PRISMA chart.RESULTS:The findings resulted in three main categories of structure, process, and outcome; 16 subcategories; and 166 codes. The category of structure included the subcategories of human resources, physical space, facilities, training, needed committees and teams, financing, and implementing mental and social programs. The subcategories of process category were functional indicators, guidelines and protocols, communication, documentation, planning/coordination, time management, and monitoring. The subcategories of behavioral-therapeutic, satisfaction, and educational outcomes were associated with the outcome category.CONCLUSION:Application of the structure, process, and outcome indicators, derived from the findings of this study, will greatly improve evaluation of the international mental health programs in schools.

An Overview of Post-Secondary Mental Health on Campuses in Ontario: Challenges and Successes

With the growing number of initiatives on campus that support student well-being, an overview of some of the systemic, sector-wide issues on campuses will help to further understanding of post-secondary student mental health needs and support improved outcomes for mental health on campus. This paper will examine systemic issues in the post-secondary mental health sector, including the challenges of maintaining best practices; review how the Mental Health Innovation Fund projects and project outcomes worked toward addressing those challenges; and highlight standards in post-secondary mental health program evaluation.

Monitoring and evaluation of mental health and psychosocial support programs in humanitarian settings: a scoping review of terminology and focus

BackgroundMonitoring and evaluation of mental health and psychosocial support (MHPSS) programs is critical to facilitating learning and providing accountability to stakeholders. As part of an inter-agency effort to develop recommendations on MHPSS monitoring and evaluation, this scoping review aimed to identify the terminology and focus of monitoring and evaluation frameworks in this field.MethodsWe collected program documents (logical frameworks (logframes) and theories of change) from members of the Inter-Agency Standing Committee Reference Group on MHPSS, and systematically searched the peer-reviewed literature across five databases. We included program documents and academic articles that reported on monitoring and evaluation of MHPSS in low- and middle-income countries describing original data. Inclusion and data extraction were conducted in parallel by independent reviewers. Thematic analysis was used to identify common language in the description of practices and the focus of each monitoring and evaluation framework. Logframe outcomes were mapped to MHPSS activity categories.ResultsWe identified 38 program documents and 89 peer-reviewed articles, describing monitoring and evaluation of a wide range of MHPSS activities. In both program documents and peer-reviewed literature there was a lack of specificity and overlap in language used for goals and outcomes. Well-validated, reliable instruments were reported in the academic literature, but rarely used in monitoring and evaluation practices. We identified six themes in the terminology used to describe goals and outcomes. Logframe outcomes were more commonly mapped to generic program implementation activities (e.g. “capacity building”) and those related to family and community support, while outcomes from academic articles were most frequently mapped to specialized psychological treatments.ConclusionsInconsistencies between the language used in research and practice and discrepancies in measurement have broader implications for monitoring and evaluation in MHPSS programs in humanitarian settings within low- and middle-income countries. This scoping review of the terminology commonly used to describe monitoring and evaluation practices and their focus within MHPSS programming highlights areas of importance for the development of a more standardized approach to monitoring and evaluation.

Qualitative Clinical Mental Health Program Evaluation: Models and Implications for Counseling Practitioners and Educators

Qualitative program evaluation is an effective tool allowing agencies and clinicians to provide holistic validation of innovation in clinical settings. Counselor educators can introduce models of qualitative program evaluation in both agency settings and counseling master's programs to enhance the development of clinical evaluation in the community and to integrate clinical and evaluative identities in clinical mental health practitioners and students. This article defines qualitative program evaluation and logic models and introduces methodologies that may be effective in clinical settings. Developing a logic model provides a structure that guides qualitative inquiry, and general categories of logic models are defined. Specific qualitative program evaluation models appropriate to clinical settings are also identified and defined. Recommendations for incorporating qualitative program evaluation into clinical practice and counselor education programs are discussed.

The effect of a community mental health training program for multidisciplinary staff

BackgroundPrimary health workers play a critical role in providing health education to people with mental disorders. In China community health workers working with people with mental health problems lack experience and training in this area. Additionally, coordination between hospital and community staff is not well established. The aim of this study was to provide an interdisciplinary community mental health training program and to evaluate the effect of the training on staff knowledge about mental health and confidence in their roles. MethodsA three-day community mental health training program was offered specifically for interdisciplinary mental health professionals. Using a one-group pre-test post-test design, participants completed a self-assessment of mental health concepts and program evaluation which included asking participants to rate their satisfaction using a five-point Likert scale and to respond to open-ended questions. ResultsForty-eight participants including health professionals from colleges, hospital and community health centers were recruited. Only 8.7% of participants had ever received community mental health training. Post-test evaluation demonstrated improvements in knowledge, and most participants were very satisfied with the program. ConclusionThe findings indicate that this brief interdisciplinary training program had a positive effect in improving knowledge about community mental health concepts and confidence in dealing with people with mental health disorders for multidisciplinary staff working in primary health care areas.

Articulación entre el trabajo clínico y la perspectiva comunitaria: un desafío actual para la salud mental en el contexto de la atención primaria de la salud

Objective: to characterize the intervention strategies used by psychologists in primary health care centers and hospitals of the Metropolitan Sanitary District of Neuquén, Argentina, within the context of primary health care. This study shows the theoretical views influencing these strategies and describes the main facilitating factors and obstacles these professionals faced in their PHC work during 2010. Methodology: the theoretical perspective used in this study takes elements from the evaluation of community mental health programs, concepts from PHC and its links to mental health and contributions from Latin American community psychology. Data were collected by reviewing existing health records, applying surveys and performing interviews. Results: The results show that psychologists develop multiple and varied intervention strategies and that, although all of them fall under the principles of PHC, they focus specifically on care, thus setting aside other dimensions such as health promotion, disease prevention, etc. Discussion and conclusions: In this context, it is postulated that one of the main challenges for mental health professionals working in the context of primary health care is bringing together clinical work and the community perspective rather than contrasting them, which is the usual approach.

School‐Based Mental Health Program Evaluation: Children's School Outcomes and Acute Mental Health Service Use

This study examined the impact of school-based mental health programs on children's school outcomes and the utilization of acute mental health services. The study sample included 468 Medicaid-enrolled children aged 6 to 17 years who were enrolled 1 of 2 school-based mental health programs (SBMHs) in a metropolitan area sometime during school year 2006-2007. A multilevel analysis examined the relative effects of SBMHs on children's absence, suspension, grade promotion, use of acute mental health services, as well as the association of child and school-level factors on the outcomes of interest. Little change in average number of days absent per month and no significant change in the use of acute mental health services were found. The mean number of days suspended per month out-of-school decreased from 0.100 to 0.003 days (p < .001). The percentage of children promoted to the next grade increased almost 13% after program enrollment (p < .01). Program type did not predict outcome changes except grade promotion. Despite the positive effect of school-based mental health programs on some school outcomes, the lack of difference between programs suggests the need to identify active mechanisms associated with outcome to make the delivery of care more efficient.

Contemporary Mental Health Program Implementation and Evaluation

Evaluation is a burgeoning field of health care program research. Experimental designs and meta-analyses have traditionally been considered the most suitable means of assessing program effectiveness. Qualitative methods also have been widely adopted. Mixed-methods designs have now emerged as a promising alternative. This paper highlights the key features of contemporary mental health program implementation and evaluation. Realistic Evaluation is identified as a relatively new methodology that is more concerned with a deeper process-oriented inquiry and appreciates that any program must be understood within the given context. Two projects applying Realistic Evaluation are cited to illustrate the value of this methodology to mental health program research.

Theoretical basis and program design of a school-based mental health intervention for traumatized immigrant children: a collaborative research partnership.

This article describes a collaborative research model for school-based mental health services that targets children who are recent immigrants with violence-related mental health symptoms. The model describes a conceptual framework used in the establishment of an academic-community partnership during the development, evaluation, and implementation of the Mental Health for Immigrants Program (MHIP), a school-based mental health intervention. The article discusses the challenges that occurred and provides specific examples of how a participatory research partnership may work together through all program phases--design through implementation and program evaluation--to meet a specific community's needs and produce generalizable knowledge. The challenges and limitations of collaborative research approaches also are discussed, with particular emphasis on the role of participatory research in the development and evaluation of school-based mental health programs.

Use of Focus Groups as a Tool for Evaluating Programs for Children and Families

Focus groups are an effective method for examining stakeholder perceptions of mental health programs for children and their families. This article presents ideas about how to use focus groups and discusses the advantages and challenges of using focus groups to evaluate programs that provide services for children. Focus groups can be implemented to evaluate many aspects of programs, including assessing stakeholder needs and examining their views of program benefits and areas for improvement. The growing emphasis on including consumer perceptions in mental health program development and evaluation will no doubt go along with the increased use of focus groups in years to come.

Mental Health in the Schools: “Linking Islands of Hope in a Sea of Despair”

The current and future status of the school psychologist's role in mental health programming is addressed. Despite concerns that school psychologists are spending the majority of their time conducting psychoeducational assessments, there is evidence that some school psychologists are conducting mental health programming. We describe “islands of hope”—programs in which school psychologists assume key roles in mental health program development, implementation, and evaluation. We conclude with suggestions for ways in which school psychologists can facilitate their participation as mental health service providers, by creating information networks, developing prereferral assistance teams, developing collaborative systems of care, demonstrating accountability, extending work into the community, becoming culturally competent, developing practitioner-university partnerships, and broadening roles and functions.

A public-academic approach to designing a state mental health program evaluation

This article describes a collaboration between a state legislature and a state university to evaluate the impact of a legislatively-mandated reorganization of the state's mental health service delivery system for adults. The authors address political, logistical, and methodological issues associated with the collaborative evaluation's design and implementation.

Administrative issues in the implementation and evaluation of community mental health programs: A case study

Much of the work that has been done in program evaluation has been in areas other than mental health such as job training programs, public assistance services, and education. These activities pose fewer problems for evaluation than do mental health services because of their relatively lower degree of ambiguity and their more concrete objectives. Community mental health programs pose special problems for evaluation because of their multiple and, at times, conflicting objectives. Contemporary community as well as inpatient programs may serve to protect people whose behavior may be dangerous to themselves, to protect others from such behavior, to cure or at least manage problematic behavior and "symptoms" of mental illness, as the mental health literature describes some behavior, and to provide basic living arrangements for

Internal working models of caregiving and security of attachment at age six

This study describes the relation between internal working models of caregiving, child attachment, and maternal behavior in the home. Thirty-two mothers of 6-year-old children were observed in the home and subsequently interviewed regarding experiential and affective dimensions of parenting. Interviews were examined in order to assess the quality of the mother's thinking regarding two dimensions of caregiving (secure base, competence) which we hypothesized to be related to attachment security. Results indicated a strong correspondence between internal working models of caregiving and child mental representations of attachments as measured from the child's response to a laboratory reunion. The correspondence between mental representations of caregiving and maternal behavior in the home was limited. Representation ratings were most strongly associated with competence-supporting behavior. Implications for infant mental health research and program evaluation are discussed.

Mental health program evaluation and needs assessment

AbstractThe course of evaluation in mental health has been altered by the sociopolitical context of the 1970s and 1980s. Among the lessons learned are (1) the limitations of some evaluation technologies, (2) improvements that have occurred in others, and (3) the need for “echelon‐specific” evaluations. Insufficient recognition has been given to the need for both internal and external evaluation and replication.