Counseling Proficiency Research Articles

Evaluating Language Models for Assessing Counselor Reflections

Reflective listening is a fundamental communication skill in behavioral health counseling. It enables counselors to demonstrate an understanding of and empathy for clients’ experiences and concerns. Training to acquire and refine reflective listening skills is essential for counseling proficiency. Yet, it faces significant barriers, notably the need for specialized and timely feedback to improve counseling skills. In this work, we evaluate and compare several computational models, including transformer-based architectures, for their ability to assess the quality of counselors’ reflective listening skills. We explore a spectrum of neural-based models, ranging from compact, specialized RoBERTa models to advanced large-scale language models such as Flan, Mistral, and GPT-3.5, to score psychotherapy reflections. We introduce a psychotherapy dataset that encompasses three basic levels of reflective listening skills. Through comparative experiments, we show that a finetuned small RoBERTa model with a custom learning objective (Prompt-Aware margIn Ranking (PAIR)) effectively provides constructive feedback to counselors in training. This study also highlights the potential of machine learning in enhancing the training process for motivational interviewing (MI) by offering scalable and effective feedback alternatives for counseling training.

Assessing nutrition literacy and nutrition counseling proficiency following an interdisciplinary culinary medicine elective.

Culinary medicine (CM) is a growing field of education that aims to bridge the gap between the clinical need for nutritional counseling and the lack of education on the topic. Healthcare professionals can aid in nutrition-related noncommunicable disease (NCD) prevention by improving a patient's dietary behavior. However, the presence of nutrition education in healthcare curricula is lacking. Early evidence indicates that CM could address this gap. The objectives of this study are to determine ifthe provision of an interdisciplinary CM elective will improve student knowledge and confidence with counseling on nutrition and culinary principles, and to improve personal dietary habits of students. This was a one-group pretest-posttest quasi-experimental design. First- and second-year osteopathic medical students (OMS) and nurse practitioner students were recruited to participate in a CM elective via email. Participants were excluded if they were not in good academic standing at their respective institutions. Twelve individuals (n=8 medical; n=4 nursing) were enrolled in the course. Participants completed pre- and postcourse surveys to determine changes in nutrition literacy (Nutrition Literacy Assessment Instrument [NLit42]), nutrition counseling proficiency (Nutrition Survey for Family Practitioners), and dietary quality (Automated Self-Administered 24-h dietary assessment tool; ASA24®). A two-sided, paired t test was conducted to determine changes in outcome variables. All 12 participants completed the precourse assessments, and 8 participants completed the postcourse assessments. Culinary activity attendance was 94.5 %. Participants exhibited a statistically significant increase in their overall nutrition literacy scores after completing the CM elective (p=0.006). Literacy subcategories indicated that the improvement came from the participant's ability to understand household measurements (p=0.005) better. Increases in self-reported proficiency were observed for participants' confidence to counsel on nutrition and prevention/wellness (p=0.02) and macronutrients in health and food safety (p=0.01). No statistically significant changes in the personal dietary pattern or quality were observed. The interdisciplinary CM elective improved nutrition literacy and some aspects of counseling proficiency. Although small shifts in dietary variables were observed, the elective did not statistically improve participants' dietary pattern. However, some changes that were observed may lead to clinically relevant outcomes if maintained long-term. These findings are encouraging. Implementing CM as an educational tool could improve healthcare practitioners' ability to understand and counsel patients on nutrition to prevent the nutrition-related NCDs.

Associations between medical students\u2019 beliefs about obesity and clinical counseling proficiency

BackgroundDespite evidence that biological and genetic factors contribute strongly to obesity, many healthcare providers still attribute obesity more to controllable behavioral issues rather than factors outside a person’s control. We evaluated whether medical school students’ beliefs about obesity correlate with ability to effectively counsel patients with obesity.MethodsClerkship-year medical students at NYU School of Medicine completed an Objective Structured Clinical Experience (OSCE) that tests ability to effectively counsel standardized actor-patients with obesity. We surveyed these students to evaluate their beliefs about the causes of obesity and their attitudes towards people with obesity. We analyzed correlations between student beliefs, negative obesity attitudes, and OSCE performance.ResultsThe response rate was 60.7% (n = 71). When asked to rate the importance of individual factors, students rated controllable factors such as unhealthy diet, physical inactivity, and overeating as more important than genetics or biological factors (p < 0.01). Believing obesity is caused by uncontrollable factors was negatively correlated with obesity bias (r = − 0.447; p < 0.0001). Believing that obesity is caused by factors within a person’s control was negatively correlated with counseling skills (r = − 0.235; p < 0.05).ConclusionsAttribution of obesity to external factors correlated with greater ability to counsel patients with obesity, suggesting that educating providers on the biological causes of obesity could help reduce bias and improve provider care.

Technology to Support Motivational Interviewing

PurposeThis paper reports the findings of motivational interviewing (MI) training with and without technology support on school-based health center (SBHC) providers' satisfaction with MI training, providers' self-report of behavioral counseling related to childhood overweight/obesity, and parents' perception of care after training. Design and MethodsThe effects of training and technology on MI is part of a larger comparative effectiveness, cluster randomized trial. Twenty-four SBHCs in six states received virtual training on MI. Half the sites received HeartSmartKids™, a bilingual (English/Spanish), decision-support technology. The technology generated tailored patient education materials. Standard growth charts were plotted and health risks were highlighted to support MI counseling. The results of the MI training included provider satisfaction with MI training and parent assessment of the components of MI in their child's care. Providers and parents were surveyed at baseline, after training, and six months after training. ResultsProviders were satisfied with training and reported improvements in counseling proficiency (p<0.0007) and psychological/emotional assessment (p=0.0004) after training. Parents in the technology group reported significant improvement in provider support for healthy eating (p=0.04). ConclusionVirtual training has the potential of preparing providers to use MI to address childhood obesity. Technology improved parent support for healthy eating. Future research should evaluate the impact of technology to support MI on patient outcomes. Practice ImplicationsChildhood obesity guidelines emphasize that MI should be used to promote healthy weight in children. Training providers on MI may help more providers incorporate obesity guidelines in their practice.

Developing Spiritual Competencies in Counseling: A Guide for Supervisors

The recently established competencies for addressing spiritual and religious issues in counseling (Association of Spiritual, Ethical, and Religious Values in Counseling, ) provide guidelines for counselors to implement in conjunction with evidence‐based counseling practices. These spiritual competencies complement the ethics code established by the American Counseling Association () in promoting increased cross‐cultural sensitivity and proficiency in counseling. Supervisory tools that promote the ethical and spiritual development of counseling trainees are identified and described.

The Development and Validation of the Comprehensive Counseling Skills Rubric

The authors provide psychometric support for the comprehensive counseling skills rubric (CCSR). The CCSR is designed to assess trainee’s counseling proficiency, identify how counseling skills are used within phases of a counseling session, and evaluate the counseling relationship. The CCSR content validity was verified with an overall mean item content validity index of .81. There was a strong domain-level internal consistency reliability (.73 to .84). Fleiss’ κ of .70 demonstrated substantial interrater reliability. Thus, the results indicate initial support for using the CCSR as a transtheoretical instrument designed to assess a counseling trainee’s ability to execute the counseling experience.

Evaluation of an Online Tobacco Cessation Course for Health Professions Students

To implement and evaluate the impact of a semester-long, online, 1-credit elective course designed to promote tobacco cessation counseling proficiency among health professions students. Online technology was used to create an elective course devoted to tobacco cessation, modeled closely after the Rx for Change curriculum. Students from pharmacy, nursing, and other health disciplines enrolled in the course. Students completed pretraining and posttraining survey instruments that assessed their self-reported skills and ability to counsel patients for tobacco cessation. Overall ability to counsel for tobacco cessation and each of the "5 A's" approach for comprehensive counseling (ask, advise, assess, assist, arrange) increased significantly from pretraining to posttraining (p < 0.001). Self-efficacy also increased from 2.2 to 4.1 (p < 0.001; on a 5-point scale). This study demonstrated that an online tobacco cessation course improved student-reported ability and skills to counsel patients on tobacco cessation.

Improving client–provider communication: evaluation of a training program for women, infants and children (WIC) professionals in New York state

Results are presented from evaluation of an intensive 1 day training program to improve the growth monitoring counseling skills of Special Supplemental Nutrition Program for Women, Infants and Children (WIC) providers. The training was framed by the patient-centered approach, and focused on a seven-step technique that emphasized eliciting client perspective on the child’s health and negotiating follow-up strategies. Changes in skill were assessed during audiotaped mock counseling sessions with simulated clients. Observed intervention effects were moderate but encouraging for future training programs. After the training, more providers elicited client perspective, and provider level of engagement in negotiating with the client increased. At post-test providers asked more open-ended questions than at pre-test, and provider-to-client talk ratio decreased. Increases in provider total and competence-related satisfaction paralleled improvements in counseling proficiency. Study results suggest that counseling skills of non-physician health providers can change after a 1 day focused training: providers were more client-centered in their discussions. Limitations and implications of the study are discussed.

Ethical and Professional Issues in Career Assessment With Lesbian, Gay, and Bisexual Persons

This article proposes a framework for the consideration of ethical and professional issues in career assessment with lesbian, gay, and bisexual persons. Themes are identified and structured based on an integration of the Division 44?Committee on Lesbian, Gay, and Bisexual Concerns Joint Task Force's "Guidelines for Psychotherapy With Lesbian, Gay, and Bisexual Clients"; the American Psychological Association's Guidelines for Multicultural Counseling Proficiency for Psychologists; and conceptual work by Prince and by Croteau, Anderson, DiStefano, and Kampa-Kokesch. The themes are organized under three domains: practice, research, and education. The first two domains are further divided into three categories: attitudes?awareness, knowledge, and skills. Applications of each theme to career assessment with lesbian, gay, and bisexual persons are discussed, drawing on relevant literature in vocational psychology.

The Development and Evaluation of a Measure of Counselor Effectiveness

The purpose of this research was to construct and evaluate a measure for assessing counselor‐trainee effectiveness. An eight‐category rating instrument was constructed and two standardized interview situations emphasizing problems commonly encountered by secondary school counselors were developed. The standardized interview situations provided by the use of coached counselees permitted a panel of six trained judges to evaluate via closed‐circuit television 30 counselor‐trainees under conditions designed to insure comparability and, hence, fair appraisal. Using a definition of reliability as the degree of consistency between judges, the average correlation between ratings given by pairs of independent raters was .50. It was concluded that the rating instrument developed and the techniques employed for its use constituted a usable, realistic measure for assessing the counseling proficiency of counselor‐trainees.