Abstract
Mental Health First Aid (MHFA) training teaches participants how to assist people experiencing mental health problems and crises. Observed behavioral assessments, post-training, are lacking, and the literature largely focuses on self-reported measurement of behaviors and confidence. This study explores the reliability of an observed behavioral assessment rubric used to assess pharmacy students during simulated patient (SP) role-play assessments with mental health consumers. Post-MHFA training, pharmacy students (n = 528) participated in SP role-play assessments (n = 96) of six mental health cases enacted by consumers with lived experience of mental illness. Each assessment was marked by the tutor, participating student, and consumer (three raters). Non-parametric tests were used to compare raters’ means scores and pass/fail categories. Interrater reliability analyses were conducted for overall scores, as well as pass/fail categories using intra-class correlation coefficient (ICC) and Fleiss’ Kappa, respectively. Test re-test reliability analyses were conducted using Pearson’s correlation. For interrater reliability analyses, the intra-class correlation coefficient varied from poor-to-good to moderate-to-excellent for individual cases but was moderate-to-excellent for combined cases (0.70; CI 0.58–0.80). Fleiss’ Kappa varied across cases but was fair-to-good for combined cases (0.57, p < 0.001). For test re-test reliability analyses, Pearson’s correlation was strong for individual and combined cases (0.87; p < 0.001). Recommended modifications to the rubric, including the addition of barrier items, scoring guides, and specific examples, as well as the creation of new case-specific rubric versions, may improve reliability. The rubric can be used to facilitate the measurement of actual, observed behaviors post-MHFA training in pharmacy and other health care curricula.
Highlights
Mental and addictive disorders affect over one billion people, globally [1]
Due to the importance of developing reliable rubrics to ensure that assessors are marking students in a reliable manner [26], this study aimed to investigate the reliability of the rubric designed to measure observed Mental Health First Aid (MHFA) skills during simulated patient role-play assessments
Suicidal thoughts and behaviors were incorporated into the scenarios as previous research has demonstrated that students often struggle to assess for suicide directly [21], and suicide education is lacking from health care curricula [7]; assessments were integrated into the curriculum to provide students with an opportunity to practice assessing for suicide, post-MHFA training
Summary
Mental and addictive disorders affect over one billion people, globally [1]. suicide is currently among the leading causes of death, worldwide [2]. Given the high prevalence of mental illness and the lack of help sought by those affected [3,4], the role of primary healthcare professionals, such as pharmacists, in providing initial care has been recognized [5,6]. Research has indicated that health professionals may lack adequate training and education in their university curricula to enable them to confidently care for people at risk of suicide [7]. Additional training may be required to equip health professionals with these skills. One such training program available internationally and increasingly embedded into university curricula [8] is Mental Health First Aid (MHFA), which teaches participants how to assess and assist those experiencing mental health problems and crises, including suicide [9]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
