Present Evaluation Data Research Articles

Networking With Industry Partners, Experiential Learning, and Teamwork: Engaging and Preparing Students Across Modalities

This case study assesses a course re-designed to scaffold a semester-long networking opportunity and experiential learning project. The course occurred online twice using Microsoft Teams in the spring and fall of 2021 and later in person once in the spring of 2023. I present quantitative student course evaluation data from all three semesters and compare them to the department, school, and university. I also address open comments from students, pedagogy, media coverage, and our fashion industry partner's perspective. Findings suggest that the course design was successful across modalities. Students rated the course highly compared to others on campus, even while online, and ratings improved over time. Student comments praised the course for offering valuable career preparation. Our industry partner also derived professional and personal value by interacting with and mentoring students. The case study concludes by encouraging educators to implement experiential learning in any modality, offering guidance, and addressing obstacles.

Supporting LGBTIQA+ communities in small rural settings: a case study of health promotion in a community health service.

People from LGBTIQA+ communities living in rural areas experience significant disparities across a range of health and wellbeing indicators, high levels of stigma and discrimination, and difficulty accessing inclusive health and support services. This article discusses a targeted health promotion intervention designed to build supportive health and social programs to address these problems. We describe the initial needs assessment undertaken to inform the intervention, the role and activities of the health promotion officer funded through the program, and present evaluation data on program outcomes. The article demonstrates that modest funding for health promotion can make a positive contribution to connecting and supporting individuals in a small regional community, leading to better health and wellbeing outcomes.

Maternal Deaths from COVID-19 in Brazil: Increase during the Second Wave of the Pandemic.

To compare death rates by COVID-19 between pregnant or postpartum and nonpregnant women during the first and second waves of the Brazilian pandemic. In the present population-based evaluation data from the Sistema de Informação da Vigilância Epidemiológica da Gripe (SIVEP-Gripe, in the Portuguese acronym), we included women with c (ARDS) by COVID-19: 47,768 in 2020 (4,853 obstetric versus 42,915 nonobstetric) and 66,689 in 2021 (5,208 obstetric versus 61,481 nonobstetric) and estimated the frequency of in-hospital death. We identified 377 maternal deaths in 2020 (first wave) and 804 in 2021 (second wave). The death rate increased 2.0-fold for the obstetric (7.7 to 15.4%) and 1.6-fold for the nonobstetric groups (13.9 to 22.9%) from 2020 to 2021 (odds ratio [OR]: 0.52; 95% confidence interval [CI]: 0.47-0.58 in 2020 and OR: 0.61; 95%CI: 0.56-0.66 in 2021; p < 0.05). In women with comorbidities, the death rate increased 1.7-fold (13.3 to 23.3%) and 1.4-fold (22.8 to 31.4%) in the obstetric and nonobstetric groups, respectively (OR: 0.52; 95%CI: 0.44-0.61 in 2020 to OR: 0.66; 95%CI: 0.59-0.73 in 2021; p < 0.05). In women without comorbidities, the mortality rate was higher for nonobstetric (2.4 times; 6.6 to 15.7%) than for obstetric women (1.8 times; 5.5 to 10.1%; OR: 0.81; 95%CI: 0.69-0.95 in 2020 and OR: 0.60; 95%CI: 0.58-0.68 in 2021; p < 0.05). There was an increase in maternal deaths from COVID-19 in 2021 compared with 2020, especially in patients with comorbidities. Death rates were even higher in nonpregnant women, with or without comorbidities.

A Mixed-Method Process Evaluation of an Intervention to Improve Social Reactions to Disclosures of Sexual Assault and Partner Abuse.

Because of the high rates and deleterious consequences of sexual assault (SA) and partner abuse (PA) on college campuses, there is a proliferation of programming to both prevent and respond to these issues. Most research to date, however, presents outcome evaluation data on these programs and neglects to present process evaluation data which are critical for program refinement and dissemination. The purpose of this study was to present process evaluation data (i.e., acceptability and feasibility) specific to a program that endeavored to increase positive and decrease negative social reactions from disclosure recipients to individuals disclosing SA and PA. Participants were 303 students who completed the program and participated in postintervention surveys and a subset of students (n = 18) who completed exit interviews. Results documented that the program was both feasible and acceptable, as evidenced by high satisfaction ratings. Important suggestions were also provided for how to improve the program, such as reducing repetition and making scenarios more realistic. Finally, participants who reported higher program engagement and more program usage generally reported more intentions to provide positive social reactions, less intentions to provide negative social reactions, and less actual negative social reactions. This information is useful not only for adapting the current program discussed herein but also for program developers and preventionists wishing to create similar programming to effectively prevent and improve response to SA and PA.

Evaluation of a Level I trauma center provider training in patient-centered alcohol brief interventions using the Behavior Change Counseling Index rated by standardized patients

BackgroundTraumatic injury requiring hospitalization is common in the USA and frequently related to alcohol consumption. The American College of Surgeons requires that Level I and II verified trauma centers implement...

Counselor Preparation for Working With At-Risk Youth

In this article, we provide an overview and evaluation of a federally funded training program designed to increase access to mental health services for at-risk children, adolescents, and transitional-age youth by expanding the number of behavioral health providers trained to provide evidence-based services. We present evaluation data and implications for counselor education and supervision.

The Development and Evaluation of a Statewide Training Center for Home Visitors and Supervisors.

This paper informs practice in community-based home visiting workforce development by describing the development and evaluation of a university-based training certificate program for home visitors and supervisors. The Interactive Systems Framework for Dissemination and Implementation (ISF; Wandersman etal., 2008) guides our conceptualization and paper organization. The ISF describes the components involved in translating research findings into effective implementation of prevention programs. We describe implementation and lessons learned from seven development activities: (a) review of the literature, (b) survey of other training initiatives across the country, (c) focus groups with home visitors and supervisors, (d) consultation with individual home visitors, (e) creation of a state advisory board of home visiting providers and stakeholders, (f) evaluation of two pilot trainings, and (g) video development. We then present evaluation data from 49 home visitors and 23 supervisors who completed the training certificate program after the pilot trainings. Both home visitors and supervisors rated training satisfaction highly, reported significant increases in self-efficacy related to the training topics, and reported extensive use of motivational communication techniques, which are the foundational skills of the training content. These and other favorable results reflect the benefits of building on advances in theory and science-based practice and of involving providers and stakeholders repeatedly throughout the development process.

Paving the Road into College and STEM for Latino Students

The purpose of this study is to thoroughly describe a program designed to strengthen the pipeline of Latino students into post-secondary science, technology, engineering, and mathematics (STEM) education, and present evaluation data to assess multiyear effectiveness. The program includes a suite of interventions aimed at students and families, and was implemented in a low-income school cluster with a high Latino population in metro Atlanta. Our intervention includes a high school and middle school mentoring program, STEM-focused extracurricular activities (summer camps, research and community service opportunities), and college and STEM career-focused community events for families. Results suggest that GoSTEM has a positive impact on students and families with respect to college and career awareness. The mentoring program components also increase students’ college readiness and self-regulatory skills at the high school level. The ex¬tracurricular programs support this effort by increasing students’ STEM-related content knowledge and learning, and their understanding about STEM careers for both high school and middle school students. Lessons learned are also presented to help guide other practitioners of Latino outreach programs.

Evaluation of a High-Engagement Teaching Program for STEM Graduate Students: Outcomes of the Future Academic Scholars in Teaching (FAST) Fellowship Program

Higher education institutions prepare future faculty members for multiple roles, including teaching. However, teaching professional development programs for graduate students vary widely. We present evaluation data from a high engagement program for STEM doctoral students. We analyzed the impact on three cohorts of participants over three academic years and identified the components most influential upon their teaching professional development. Participants found the year-long teaching assessment project and the disciplinary and reflective focus instrumental for improving their knowledge of teaching and learning. We recommend these components for the design of other such high-engagement programs.

Improving primary care prescribing safety using routine data for intervention and evaluation: four trials in 500+ practices

ABSTRACTObjectivesHigh-risk prescribing in primary care is common and causes considerable harm, with for example, ~6% of emergency hospital admissions caused by adverse drug effects. Our aim is to rigorously develop and evaluate complex interventions to improve primary care prescribing safety using routine clinical and administrative data.&#x0D; ApproachCo-design of interventions by joint academic and clinical teams with implementation embedded in NHS Scotland information and contractual systems. Routine data from general practice electronic medical records (EMRs) or from the NHS Scotland prescribing payment system are used to both underpin the interventions and allow evaluation. Two cluster-randomised trials in 295 general practices have been completed (EFIPPS and DQIP), and two trials in 298 practices are in progress (EPIPP and POEMS), involving ~55% of Scottish general practices in total. The completed EFIPPS trial was a three-arm cluster randomised trial of a routine data feedback intervention +/- a theory-informed behaviour intervention involving 262 (~95%) of practices in three health boards. The completed DQIP trial was a stepped-wedge cluster randomised trial in 33 (50%) practices in one Health Board, the intervention comprising a web-based informatics tool to identify patients needing review from practice EMRs and to facilitate and structure review, an educational outreach visit, and financial incentives to review patients. EPIPP is evaluating a more complex feedback intervention, where both arms receive an active intervention on a different topic, involving all 242 practices in one Health Board. POEMS is evaluating an ‘enhanced medication summary’ to support contractually required-polypharmacy reviews in older people taking 10 or more regular medications, involving all 56 practices in one Health Board.&#x0D; ResultsThe simpler EFIPPS intervention reduced high-risk prescribing by 12% by 15 months after feedback started. We are currently examining what happened in the year after feedback ceased. The more intensive DQIP intervention reduced high-risk prescribing by 37% at 12 months. This effect was sustained in the 12 months after the financial incentives ceased because the intervention reduced new high-risk prescribing. The DQIP intervention also led to large reductions in related emergency hospital admissions with gastrointestinal bleeding, acute kidney injury and heart failure.&#x0D; ConclusionWe have shown that informatics-based interventions using routine data and co-designed with clinicians and managers are effective in improving primary care prescribing safety. The presentation will describe our approach to intervention development and implementation, show examples of the interventions, and present outcomes and process evaluation data from the two completed trials.

Assessing and predicting shifts in mountain forest composition across 25 years of climate change

AbstractAimSpatial predictions of future communities under climate change can be obtained by stacking species distribution models (S‐SDM), but proper evaluation of community S‐SDM predictions across time with fully independent data has rarely been carried out. The aim of this study was to evaluate the predictive abilities of S‐SDMs for whole forest communities across the last 25 years in a mountain region.LocationThe western Swiss Alps.MethodsWe used past vegetation surveys (2,984 plots) and environmental data from the 1990s to calibrate SDMs for 364 plant species and predict changes in forest composition under contemporary conditions. These projections were then evaluated by resurveying a random subset of 92 forest plots in summer 2014.ResultsSpecies distribution models showed the same accuracy in the past (calibration data) and present (evaluation data). The S‐SDMs correctly predicted the general trends in species richness and shift of ecological conditions (i.e., temperature, moisture) at the regional level. However, it proved more difficult to identify precisely which forest communities or areas are most or least affected by climate change.Main conclusionOur results show that, across a period of a few decades, S‐SDMs can usefully predict trends in macroecological properties such as richness or average ecological conditions, but fail to accurately predict changes in composition. This is likely due to the combined effects of the stochasticity of local colonization and extinction events, dispersal limitations, community assembly rules (e.g., competition), observer bias, model and location errors and interannual variation. Furthermore, these models cannot account for potential species adaptations leading to persistence in sites predicted unsuitable. This highlights the need for developing more accurate forest community predictions as support to help prioritizing conservation actions.

A 25-Year Experience with a Project-Centered Master in Public Health: Key to Public Health Relevance and Educational Efficacy?

The World Health Organization has recognized the shortage of competent public health workforce as a major problem worldwide and claimed an urgent need for action. Training public health personnel, effective in an educational perspective, yet with public health relevance in mind, remains a challenge for university-based programs. We present evaluation data of the 25-year experience of a project-centered, problem-oriented, on-job Master in Public Health program at the University of Geneva. Several data collection strategies were used. First, achievement of learning objectives was investigated through content analysis of students’ reports on personal project, through problem-solving final examinations and through self-reported progress on a professional profile. Second, possible public health impact was estimated through analysis of implemented project. Third, yearly discussions with the successive cohorts use the SWOT grid (strengths, weaknesses, opportunities and threats) to investigate the students’ perception of the program. Learning objectives were mainly acquired through community health projects realization, partly also through group work, exercises, readings and lectures. Self-reported progress on a professional profile showed significant acquisitions in all tested public health competencies. All students were able to implement at least one of their planned projects. Students’ perception considered the project/problem-centered approach as positive, but underlined the importance of time constraints and heavy workload in an on-job training program. In our experience a project-centered, problem-based Master program in Public Health allowed students to achieve new competencies and communities to benefit from project implementation addressing public health issues.

A Research Preparatory Program for First-Year College Students: Student Selection and Preparation Lead to Persistence in Research

Undergraduate research experiences may increase persistence in STEM majors. We describe a research program that targets first-year students selected for their curiosity and attitudes towards science. We explain the implementation of the program over 3 years and present evaluation data using a group of matched controls. Participants and controls pursued STEM degrees at equivalent rates, but participants were significantly more involved in research. Initial laboratory interest and mentor pairing may have played a role in this finding. Female participants, particularly those with male laboratory mentors, engaged in more research than men.

The Flipped Classroom Improves Student Achievement and Course Satisfaction in a Statistics Course

There are but a handful of experimental or quasi-experimental studies comparing student outcomes from flipped or inverted classrooms to more traditional lecture formats. In the current study, I present cumulative exam performance and student evaluation data from two sections of a statistics course I recently taught: one a traditional lecture ( N = 19) and the other a flipped class ( N = 24). Independent samples t-tests revealed students in the flipped classroom outperformed their lecture peers by more than a letter grade on the final exam. Further, these students were more satisfied with the course overall, a novel finding in this burgeoning area of research. This latter point, I argue, is likely due to the strong cohesion between the in-class and out-of-class content.

Changing the culture of a medical school by orienting students and faculty toward community medicine.

Oklahoma's health status has been ranked among the worst in the country. In 1972, the University of Oklahoma established the Tulsa branch of its College of Medicine (COM) to expand the physician workforce for northeastern Oklahoma and to provide care for the uninsured patients of the area. In 2008, the Tulsa branch launched a distinct educational track, the University of Oklahoma COM's School of Community Medicine (SCM), to prepare providers equipped and committed to addressing prevalent health disparities.The authors describe the Tulsa branch's Summer Institute (SI), a signature program of the SCM, and how it is part of SCM's process of institutional transformation to align its education, service, and research missions toward improving the health status of the entire region. The SI is a weeklong, prematriculation immersion experience in community medicine. It brings entering medical and physician assistant students together with students and faculty from other disciplines to develop a shared culture of community medicine. The SI uses an unconventional curriculum, based on Scharmer's Theory U, which emphasizes appreciative inquiry, critical thinking, and collaborative problem solving. Also, the curriculum includes Professional Meaning conversations, small-group sessions to facilitate the integration of students' observations into their professional identities and commitments. Development of prototypes of a better health care system enables participants to learn by doing and to bring community medicine to life.The authors describe these and other curricular elements of the SI, present early evaluation data, and discuss the curriculum's incremental evolution. A longitudinal outcomes evaluation is under way.

A Community Engaged Curriculum for Public Service Psychiatry Fellowship Training

Transforming the mental health system into a recovery oriented, integrated system of care requires a psychiatric work force that understands the relationship between recovery processes and community living. Fellowship programs in public and community psychiatry contribute to this transformation by educating psychiatrists about recovery, system dynamics, leadership, effective administration and community involvement. This paper describes a novel approach to fellowship programming that accomplishes these aims through an organizational strategy that emphasizes community engagement. After describing the administrative background for the program, we describe how the content curriculum and teaching process focus on the engagement of community members-both service users and service providers-as participating faculty. The faculty includes over 100 consumers, family members, advocacy group representatives, clinicians, and administrators. We present evaluation data obtained from 45 of the 100 community and university faculty who participated in the first 2years' of the fellowship and conclude with a critique and recommendations for further progress in community engaged fellowship training.

Bridging the gap: How is integrating communication skills with medical content throughout the curriculum valued by students?

ObjectiveTo describe a program with integrated learning of communication and consultation skills developed with the intention of preventing deterioration of communication skills, and to present student evaluation data. MethodsDescription and evaluation of the program through: (1) monthly student evaluations; (2) questionnaire on student perceptions about the integrated curriculum; (3) a questionnaire about the value of one specific integrated training preceding the pediatric clerkship. ResultsKey components of training throughout years 3–6 are reinforcement throughout the clinical years, adapting communication training to the clinical context of clerkships using a sandwich model with cycles of preparation, clerkship, and reflection. Evaluation: response rates were 69%, 93% and 93%, respectively. Students value practicing integration of communication and medical content with SPs who represent the population of their next clerkships. They appreciate the multisource feedback during the training, feedback by clerkship specific specialists and SPs is valued most. ConclusionsThis description shows an example of an integrated curriculum that helps students to feel well prepared for their communication tasks in subsequent clerkships. Practice implicationsDesigning and implementing communication curricula to address the issue of integration is feasible. The effects of such integrated programs should be subject to future studies.

Globalization as Re‐traumatization: Rebuilding Haiti from the Spirit Up

This article provides a case study of Haiti within the context of both its distant history of colonialism and its recent history of globalization. We then provide a discussion that highlights the relationship between globalization, poverty, and mental health in low and middle‐income countries and describe the development of Pwogwam Santé Mantal, a community‐based mental health program in Jeremie, Haiti. We present results of focus groups and interviews that provide a sense of how mental health issues are discussed in this region and present results from a pilot survey which corroborate focus group and interview data. We describe a mental health seminar that took place in July of 2010 and present outcome evaluation data for the seminar. We end with a discussion of how these data and this mental health program address the problematic relationship between globalization and mental health and highlight implications for future research and program/policy development.

The development of an adolescent smoking cessation intervention--an Intervention Mapping approach to planning

The objective of this project was to develop a theory- and evidence-based adolescent smoking cessation intervention using both new and existing materials. We used the Intervention Mapping framework for planning health promotion programmes. Based on a needs assessment, we identified important and changeable determinants of cessation behaviour, specified change objectives for the intervention programme, selected theoretical change methods for accomplishing intervention objectives and finally operationalized change methods into practical intervention strategies. We found that guided practice, modelling, self-monitoring, coping planning, consciousness raising, dramatic relief and decisional balance were suitable methods for adolescent smoking cessation. We selected behavioural journalism, guided practice and Motivational Interviewing as strategies in our intervention. Intervention Mapping helped us to develop as systematic adolescent smoking cessation intervention with a clear link between behavioural goals, theoretical methods, practical strategies and materials and with a strong focus on implementation and recruitment. This paper does not present evaluation data.

Effective management of roles and responsibilities: Driving accountability in software development teams

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Teams differ in the way they define roles and responsibilities and in the level of formalism by which they establish and communicate these definitions. Responsibilities are not always clearly defined, and there is often confusion or mismatch between the individual's perspectives of a role and the expectations of that role by other team members. This lack of shared understanding can lead to issues in performance and lack of accountability. We review the notion of role specification as part of software development governance and present an approach for specifying responsibilities in terms of decisions to be made during the life cycle of software development artifacts. We present evaluation data from software teams as they redefine their roles and shape their responsibilities. We further present a tool for governance specification based on this approach, which can ensure that these specifications are adhered to in the software development platform. We conclude by describing a methodology for how the tool and approach can be implemented to help software development teams understand and evolve the appropriate governance for their needs. </para>