Program Evaluation Research Articles

Development and Evaluation of a Presenter Coaching Program at an Academic Surgical Conference.

Development and Evaluation of a Presenter Coaching Program at an Academic Surgical Conference.

Preliminary evaluation of a mindfulness intervention program in women with long COVID dysautonomia symptoms.

Preliminary evaluation of a mindfulness intervention program in women with long COVID dysautonomia symptoms.

Program evaluation of internet-delivered cognitive behavioral treatments for anxiety and depression in a digital clinic

Program evaluation of internet-delivered cognitive behavioral treatments for anxiety and depression in a digital clinic

Process evaluation of quality of precancerous cervical lesion screening program in selected public health centers in Addis Ababa, Ethiopia.

Process evaluation of quality of precancerous cervical lesion screening program in selected public health centers in Addis Ababa, Ethiopia.

Development and Evaluation of Interprofessional Education Programs for Nursing and Health-related Students

Development and Evaluation of Interprofessional Education Programs for Nursing and Health-related Students

Development and applicability evaluation of a nutrition education program for residents and users of disability social welfare facilities in Korea: a mixed-methods study

Objectives: This study aimed to develop a nutrition education program based on social cognitive theory to promote the health of individuals using facilities for people with disabilities. It also sought to evaluate the applicability of the educational materials through assessments by counselors at the Social Welfare Food Service Management Support Center.Methods: A group of six experts developed the program based on a needs assessment of nutrition education in facilities for individuals with disabilities. Applicability was evaluated through an online survey of 26 counselors from Social Welfare Food Service Management Support Centers nationwide in July 2023, and the results were analyzed.Results: The nutrition education program includes a basic course on personal hygiene, dining etiquette, picky eating prevention, and obesity management. The advanced course covers dietary management for chronic diseases, such as meal planning for hypertension, diabetes management, and dietary principles for dysphagia. Additionally, lecture PPTs, individual activity sheets, and experiential teaching aids were developed. Applicability evaluations showed high scores, with the teaching-learning plan and PPT averaging 4.15 and the experiential teaching aids scoring 4.17, all above 4.0.Conclusion: This study developed a nutrition education program for individuals with disabilities and assessed its applicability and usability. Implementing this program in disability welfare institutions could enhance health promotion and improve the quality of life for individuals with disabilities.

예비유아교사의 PBL과 디자인씽킹에 기반한 영유아프로그램 개발 경험

The purpose of this study is to explore the experience of developing an early childhood program that based on PBL and design thinking for pre-service early childhood teachers and to examine its meaning. In this study, 25 students in the 3rd grade of the Department of Early Childhood Education at S University, located in the K area, were selected as participants to participate in the development of PBL and design thinking-based early childhood programs in early childhood program development and evaluation classes for a total of 15 weeks from September to December 2022. As a result of the study, the experience of the PBL-based early childhood program development process appeared as 'kindergarten built on white paper' and 'the result of constant questions', and the experience of the design thinking-based infant program development process was analyzed as 'empathy that changed the perspective' and 'the challenge of realizing the imagination in the head'. This study is significant in that it revealed that an early childhood program development class that based on PBL and design thinking can be an effective way to improve the power, empathy, and cooperative problem-solving skills required for early childhood teachers.

Exploring Healthcare Students’ Perceptions about Race and Privilege at a Large Academic Medical Center: A Pilot Study of a Peer Education Program

Amid the racial unrest stemming from the summer of 2020, in the United States, many institutions had difficult conversations about race and the impact of racism with employees, students, and patients. In January 2021 and February 2021, we recruited undergraduate students for the Peer Education Program (PEP), a voluntary program conducted at a large academic medical center, and encouraged students to have a safe space to discuss controversial topics in healthcare. From April 2021 to July 2021 four interdisciplinary healthcare students met once a month for 1 hr to discuss privilege, race/racism, leadership, and advocacy and their influence on health equity. Four students completed PEP, and three students completed the program evaluation survey. Conventional content analysis was used to code data and identify major themes to evaluate PEP. Students described privilege in a positive context, using words like “freedom,” “special right/advantage,” and “opportunity.” Race was viewed as (1) a way to discriminate, (2) a “man-made concept” used to establish racial hierarchy, and (3) insignificant due to the complexity of human beings. Students realized there are several required steps to adopt and sustain social change in healthcare, which led to the “The Four A’s of Social Change”: Awareness, Acknowledgment, Action, and Advocacy. Program strengths included topics, guest speakers, and open dialogue. Recommended changes included allocating more time for dialogue and increasing the frequency of monthly meetings. To be an influential and empathetic leader in healthcare, one must understand the impact that privilege and race/racism have on patients and equitable healthcare.

Development of a Program Evaluation Framework for Improving the Quality of Undergraduate Medical Education

The development and implementation of a systematic program evaluation framework is critical for improving the quality of the undergraduate medical education. At Yonsei University College of Medicine, we established the following five evaluation domains that encompass various aspects of educational experiences: (1) preclinical curriculum; (2) clinical curriculum; (3) educational environment, resources, and systems; (4) performance of students and graduates and program outcomes; and (5) implementation and outcomes of the curriculum. Specific evaluation indicators were designed within these domains and validated through the Delphi technique, which integrated expert opinions. In total, 98 indicators were identified across five domains. These indicators will function as a comprehensive tool for assessing medical education programs. The proposed evaluation framework addresses both shortand long-term educational changes, facilitating systematic monitoring, continuous quality improvement of curricula, and better outcomes for students. As this framework is grounded in the unique context of the institution, it is appropriate for a comprehensive evaluation of interactions at various educational stages. Furthermore, it may serve as a strategic foundation for identifying areas that require improvement, ensuring that the curriculum aligns with current medical education standards and practices. The framework’s structured approach and continuing evaluation processes may make it possible to obtain essential data for ongoing development, potentially contributing to a robust system for quality improvement in medical education. The findings of this study are expected to serve as a valuable reference for developing similar evaluation frameworks in other medical schools.

Community as Medicine: A Novel Approach to Improve Health Behaviors and Mental Well-Being for Vulnerable Populations.

Evaluate a group-based, trauma-informed health coaching model based on Lifestyle Medicine pillars, delivered by community health coaches, to improve health behaviors and mental well-being for vulnerable populations. This program evaluation analyzed quantitative longitudinal data with linear mixed models and qualitative data with reflexive thematic analysis. Participants (n = 720) were low-income adults referred through Federally Qualified Health Centers (FQHCs), who participated in weekly 90-120 minute groups for 3 months. Data were collected via monthly surveys, including the PHQ-9, GAD-7, UCLA 3-item loneliness, exercise as a vital sign, a 2-item dietary screener, and 3 qualitative questions. Participants saw significant reductions in depression, anxiety, and isolation, and significant increases in daily servings of fruits and vegetables, and weekly minutes of exercise. The qualitative analysis identified 4 themes related to the drivers and reinforcers of positive behavior change and improved mental well-being. Initial data suggest this model helps reduce depression, anxiety, and isolation, and promotes positive behavior change within populations most impacted by health inequity. Qualitative results identified drivers of positive change, such as creating a sense of belonging and mutual support. Future research should consider how to continue to scale this program to a variety of populations and across different settings.

Determinants of survival in women diagnosed with breast cancer between 2008 and 2017: An analysis of a cohort using data from four Population-Based Cancer Registries of Colombia.

Determinants of survival in women diagnosed with breast cancer between 2008 and 2017: An analysis of a cohort using data from four Population-Based Cancer Registries of Colombia.

Mobilizing Health Promotion Through Canada's Student Mental Health Network: Concurrent, Mixed Methods Process Evaluation.

Mental health issues among Canadian postsecondary students are prevalent. In tandem, an increased acknowledgment of the need for upstream mental health support has been highlighted. While the majority of institutions offer some form of mental health promotion, research suggests students are failing to access support due to barriers including lack of awareness, geographical and financial barriers, and lack of relevance in offerings. Canada's Student Mental Health Network is a web-based knowledge mobilization initiative designed to fill these gaps. With content created and curated "for-students, by-students" and reviewed by subject matter experts, the Network serves as a one-stop shop for evidence-based, mental health support for postsecondary students. The goal of this research was to conduct the first component of a comprehensive program evaluation of the Network. This paper details a formative, process evaluation after approximately 1 year of operations, with the goal of assessing acceptability and feasibility. Using a concurrent mixed methods study design, quantitative and qualitative data were simultaneously collected from students in order to evaluate the acceptability and feasibility of the Network as a mental health promotion resource. Quantitative data were automatically collected through Google Analytics via the website over the course of the first year of operations. Data collected included the number of users accessing the website, user engagement, and user "stickiness." Quantitative data were used to evaluate both accessibility and feasibility. Qualitative data were collected via individual, digital interviews conducted with a modest sample of students (n=8) across areas and levels of study. Qualitative data derived more detailed insights into user experience and website attributes, as well as feedback on content delivery, providing evidence used to evaluate feasibility. A total of 1200 users globally accessed the Network within the first year of operations, with Canadian users accounting for nearly 90% of total website traffic. An overall 66% engagement rate was observed, with the average user visiting 7 pages per session. Further support for the acceptability of the Network is demonstrated in the Canada-wide reach of the content development and review team. Evidence for the feasibility of the Network was observed through website use statistics indicating the most frequently viewed pages aligned with our goals: providing mental health education and increasing awareness of available resources. Qualitative feedback provided additional context surrounding the feasibility of the space, including positive feedback on the esthetics, relevance, usability, inclusion, and accessibility. Areas for content expansion and improvements to accessibility were also identified. The results of this study provide evidence in support of the feasibility and acceptability of the Network as a web-based knowledge mobilization initiative in support of postsecondary students' mental health and well-being. Future research will pursue a summative, impact assessment to evaluate utility.

Evaluation of Universal Screening for Substance Use Risk on a Labor and Delivery Unit in Rural Pennsylvania.

Evaluation of Universal Screening for Substance Use Risk on a Labor and Delivery Unit in Rural Pennsylvania.

Evaluation of the Power Up program: a health promotion program encouraging healthy lifestyle habits among youth in summer day camps—study protocol

Evaluation of the Power Up program: a health promotion program encouraging healthy lifestyle habits among youth in summer day camps—study protocol

Evaluation of the social inclusion programme in Slovenia

Evaluation of the social inclusion programme in Slovenia

Peak Team: Survey of Student Strengths and Needs and Teacher’s Evaluation of an Innovative Trauma-Informed School Program

Peak Team: Survey of Student Strengths and Needs and Teacher’s Evaluation of an Innovative Trauma-Informed School Program

Content Analysis of Social Determinants of Health Accelerator Plans Using Artificial Intelligence: A Use Case for Public Health Practitioners.

Public health practice involves the development of reports and plans, including funding progress reports, strategic plans, and community needs assessments. These documents are valuable data sources for program monitoring and evaluation. However, practitioners rarely have the bandwidth to thoroughly and rapidly review large amounts of primarily qualitative data to support real-time and continuous program improvement. Systematically examining and categorizing qualitative data through content analysis is labor-intensive. Large language models (LLMs), a type of generative artificial intelligence (AI) focused on language-based tasks, hold promise for expediting content analysis of public health documents, which, in turn, could facilitate continuous program improvement. To explore the feasibility and potential of using LLMs to expedite content analysis of real-world public health documents. The focus was on comparing semiautomated outputs from GPT-4o with human outputs for abstracting and synthesizing information from health improvement plans. Our study team conducted a content analysis of 4 publicly available community health improvement plans and compared the results with GPT-4o's performance on 20 data elements. We also assessed the resources required for both methods, including time spent on prompt engineering and error correction. Accuracy of data abstraction and time required. GPT-4o demonstrated abstraction accuracy of 79% (n=17 errors) compared to 94% accuracy by the study team for individual plans, with 8 instances of falsified data. Out of the 18 synthesis data elements, GPT-4o made 9 errors, demonstrating an accuracy of 50%. On average, GPT-4o abstraction required fewer hours than study team abstraction, but resource savings diminished when accounting for time for developing prompts and identifying/correcting errors. Public health professionals who explore the use of generative AI tools should approach the method with cautious curiosity and consider the potential tradeoffs between resource savings and data accuracy.

Retrospective Evaluation of Fluoride Varnish Application Programs and Their Impact on Early Childhood Caries

Retrospective Evaluation of Fluoride Varnish Application Programs and Their Impact on Early Childhood Caries

A avaliação de revistas jurídicas no Brasil: perspectivas futuras em um cenário “pós-Qualis”

The extinction of “Qualis Periódicos” by CAPES marks a transformation in the evaluation of scientific article production in Brazil, particularly in the field of Law. For years, Qualis was the main (and perhaps the only) criterion for classifying law journals in Brazil, influencing both scholars’ choice of periodicals and the evaluation of stricto sensu Graduate Programs. The system brought significant advancements, such as increased objectivity in evaluation, greater recognition of scientific journals, and the strengthening of peer review. However, it also faced criticism, including a lack of predictability in the early evaluation cycles, delays in classification, and the imposition of uniform criteria that were inadequate for certain fields. With CAPES' new methodology, the evaluation aims to focus on individual articles rather than just the journals that publish them. Each evaluation area will have the autonomy to adopt different criteria, combining bibliometric indicators, qualitative analysis of articles, and qualitative criteria for journals. In the case of Law, indexing in recognized databases such as Scopus, Web of Science, SciELO, and Redalyc emerges as a relevant parameter for assessing publication quality. The adaptation of law journals to the requirements of these indexing databases becomes essential to expanding the number of indexed periodicals. Measures such as improving editorial practices and adhering to international indexing standards are fundamental to ensuring the continued recognition of law journals within CAPES' new evaluation framework.

Evaluation of PT Petrokimia Gresik's LONTAR Program With SROI (Social Return On Investment)

This study evaluates the LONTAR Program (Kelompok Jajanan Masyarakat Sekitar) by PT Petrokimia Gresik in 2023 using the Social Return On Investment (SROI) method. This program aims to empower MSMEs through skills training and free bazaar facilities. SROI analysis is carried out using the Present Value, Future Value, and Real Value approaches to measure social, economic, and environmental impacts. The results show SROI values of 2.20 (Present Value), 2.07 (Future Value), and 2.08 (Real Value), which means that every investment of IDR 1.00 generates a social return of IDR 2.20, IDR 2.07, and IDR 2.08. These findings indicate a positive and sustainable impact, especially in increasing the income and skills of MSMEs. The study recommends scaling up programs and ongoing training to maximize their impact, while encouraging the adoption of SROI methods in the evaluation of corporate social responsibility programs.