Program Evaluation Research Articles

English Premier League and English Football League academy managers’ experiences of how psychosocial skills and characteristics are identified and developed in youth academy soccer players

ABSTRACT Psychological skills and characteristics (PSCs) are vital for player development, yet there is a lack of agreement on which are important and how they should be identified and developed in academy players, creating barriers to implementation into curricula. The key stakeholder with considerable insight into PSC development is the academy manager (AM). The aims of this study were (a) explore AMs’ experiences of PSC identification and development, (b) identify challenges AMs feel affect development of PSCs. Nine AMs (i.e. Category 1; n = 3, Category 2; n = 3 and Category 3; n = 3) comprising an average coaching experience of 20.4 ± 7.4 years and time in the role of 5.2 years ±3.2 years participated in semi-structured interviews. Reflexive thematic analysis generated themes identified ideal PSC outcomes that AMs felt were important. AMs identified current practices within academies alongside challenges: transitions, need to upskill coaches, access to additional support from external partners and National Governing Bodies. There was consensus that PSCs are key drivers of progression and that embedding them into the pathway was critical to facilitating development. Academies should think carefully about design, implementation and evaluation of programmes to support player PSC development for success during and after soccer careers.

A research agenda to support economic resilience in US oil and gas producing communities

Abstract Major reductions in greenhouse gas emissions will require a transition away from fossil fuels, including oil and natural gas. However, little research has examined the implications of such a transition for the workers and communities who depend on these industries to support local and regional economic wellbeing. In this perspective, we lay out a research agenda that can help inform policymakers as they seek to craft effective policies to support affected communities. We focus on the United States, the world’s largest oil and gas producer, and identify three key policy areas where new scholarship is needed to inform policymaking: economic and workforce development, public finances, and environmental remediation. Although it is not a comprehensive research agenda, we identify dozens of distinct research questions that will require a mix of methods and disciplinary lenses, including basic data gathering, community engagement, program evaluation, policy analysis, political analysis, and more. The goal of this article is to encourage scholars to take up these topics and expand them in the years ahead to ensure that oil and gas communities become more economically resilient in the face of deep uncertainty over the future of the domestic and global energy system.

Monitoring and disaster prevention of high and steep sandstone slopes along highways under construction

High and steep sandstone slopes along highway line are at high risk of disasters such as landslides, cracking of support structures, and so on. The monitoring, early warning, and emergency response of such slope disaster face enormous challenges, especially during the rainy season. In this paper, intelligent monitoring, early warning and forecasting system were carried out for the high steep sandstone slope with a transmission line tower at the slope crest along the highway under construction in Guangxi, China. The automatic monitoring data, emergency rescue program and rescue effect were analyzed, and emergency rescue measures for high steep slope protection were taken. The research results show that timely access to disaster warning information can effectively support the analysis of disaster causes and the evaluation of disposal programs. Deep-hole monitoring of deformation characteristics can determine the stable state of slopes, and the tangent angle warning criterion can be used for early warning and prediction of high steep slope landslides. By analyzing the location of the sliding surface and taking timely emergency disposal measures such as layered counterpressure method and micropipes, the landslide activities can be effectively controlled to prevent further acceleration of slope collapse. This study can provide an important reference for the monitoring, early warning, forecasting and emergency rescue of sandstone slopes along highways under construction.

Mapping the costs and socioeconomic characteristics involved in traumatic brain injuries: a scoping review.

To identify the articles in the existing literature that analyse healthcare costs according to the socioeconomic position (pre- or post-injury) for traumatic brain injury survivors. Secondary aims were to describe the types of costs and socioeconomic characteristics and to determine whether socioeconomic characteristics affect the risk of traumatic brain injury or whether the consequences of trauma alter living conditions post-injury. This scoping review followed the methods proposed by Arksey and O'Malley. The literature search was performed in 5 databases. Twenty-two articles were included, published between 1988 and 2023. Only 2 articles (9%) followed the guidelines for economic evaluation of healthcare programmes and 2 articles (9%) evaluated socioeconomic position "completely" with 3 main individual measures of socioeconomic characteristics (i.e., education, income, and occupation). The relationship between costs and socioeconomic characteristics could vary in 2 ways in traumatic brain injury: socioeconomic disadvantage was mostly associated with higher healthcare costs, and the cost of healthcare reduced the survivors' living conditions. This work highlights the need for a detailed and methodologically sound assessment of the relationship between socioeconomic characteristics and the costs associated with trauma. Modelling the care pathways of traumatic brain injury would make it possible to identify populations at risk of poor recovery or deterioration following a TBI, and to develop specific care pathways. The aim is to build more appropriate, effective, and equitable care programmes.

Feasibility and acceptability of a culinary nutrition programme for adults with mild-to-moderate intellectual disability: FLIP Food and Lifestyle Information Programme.

Culinary nutrition education can support improved diet-related health and wellbeing. This pre-post pilot study aimed to assess feasibility and acceptability of an eight-session culinary nutrition programme, the Food and Lifestyle Information Programme (FLIP), for adults with mild-to-moderate intellectual disability. A secondary aim was to evaluate preliminary programme effectiveness. Participants were recruited through a disability service provider. Feasibility measures were: recruitment and retention; implementation; engagement and participation; adverse outcomes; and feasibility of outcome measures. Acceptability was assessed using an interactive process evaluation. Effectiveness measures included cooking frequency, cooking and food skill confidence and diet quality. Six of eight participants completed the intervention with high attendance and programme engagement. FLIP was well received by participants and support workers. No adverse outcomes occurred. Diet quality was feasible to assess. Findings can inform content, delivery and evaluation of future culinary nutrition programmes for adults with mild-to-moderate intellectual disability.

Menumbuhkan Budaya Menulis melalui Pelatihan Artikel Ilmiah di Kalangan Guru Madrasah Ibtidaiyah Swasta Darul Dakwah wal-Irsyad Lapeo, Sulawesi Barat

<p><em>The role of teachers in the world of education is crucial in developing student's collaborative intelligence. The ability to write scientific articles can help teachers contribute more to disseminating their knowledge and experience. This program is designed to improve teacher's skills in writing scientific articles through the introduction of basic concepts and effective writing practices. The participants in this program are 14 teachers from Madrasah Ibtidaiyah Swasta Darul Dakwah wal-Irsyad Lapeo (MIS DDI Lapeo). The method of this program consists of the theory session and the practice session. The theory session discusses the basic elements of scientific writing, including article structure, use of references, writing ethics, and good writing techniques. Furthermore, practical sessions provide opportunities for teachers from MIS DDI Lapeo to apply the knowledge they have gained with direct guidance. Program evaluation is carried out through pre-tests and post-tests to measure the increase in participant's understanding and skills. Through this activity, teachers from MIS DDI Lapeo experienced an improvement in understanding scientific writing, as well as an enhancement in practical skills in structuring and writing high-quality scientific articles</em><em>.</em></p>

Competency based medical education implementation at the institutional level: A cross-discipline comparative program evaluation

Introduction As an early adopter of competency-based medical education (CBME) our postgraduate institution was uniquely positioned to analyze implementation experience data across programs, while keeping institutional factors constant. We described participants’ experiences related to CBME implementation across programs derived from early program evaluation efforts within our setting. Methods This evaluation focused on eight residency programs at a medium-sized academic institution in Canada. Participants (n = 175) included program leaders, faculty, and residents. The study consisted of 3 phases: (1) describing intended implementation; (2) documenting enacted implementation; and (3) comparing intended with enacted implementation to inform adaptations. Each program’s findings were summarized in technical reports which were then analyzed thematically. Cross program data were organized by themes. Results Six themes were identified. All groups emphasized the need for ongoing refinement of CBME resulting from shared tensions such as increased assessment burden. However, there were some disparate CBME-related experiences between programs such as the experience with entrustable professional activities, the interpretation of retrospective entrustment anchors, and quality of feedback. Conclusion We detected several cross-program successes and important challenges related to CBME. Our experience can inform other programs engaging in implementation and evaluation of CBME.

Re-PROGRAM: The evaluation of a brief intervention program for patients with functional seizures in an outpatient hospital setting.

Functional seizures (FS) account for 20%-25% of referrals to specialist epilepsy clinics. They are associated with major disability, increased mortality, and frequent and costly health care use. Current guidelines emphasize the importance of implementing clinical pathways to coordinate and deliver effective treatment, but there are few targeted evidence-based interventions that reliably improve patient outcomes, and treatment resources are limited. We conducted a retrospective evaluation of Re-PROGRAM, a novel, brief intervention for functional seizure patients, to assess its feasibility in an outpatient setting. Twenty-nine patients with FS undertook Re-PROGRAM between August 2020 and January 2022 at the Alfred Hospital Functional Seizures Clinic, Melbourne, Australia. The intervention comprised five 60-90-min consecutive weekly appointments via telehealth, where psychologists engaged patients in a structured program of seizure management skills, lifestyle modification, and behavioral activation strategies. Following the intervention, patient feedback was collected in routine clinical follow-up as well as with a 24-item self-report pre-/postintervention comparison questionnaire. All 29 patients who enrolled in Re-PROGRAM completed the scheduled sessions. Of those who returned the postintervention questionnaire (n = 16), 15 reported a reduction in seizure frequency. Four patients were lost to follow-up. Of the remaining nine, eight reported seizure frequency reduction during clinical follow-up. Qualitative analysis of the feedback revealed the majority of patients reported reduced seizure duration, intensity, and bothersomeness, and patients felt improvements in their sense of control over seizures, confidence to use seizure control strategies, assertive communication, problem solving, coping skills, relationships with others, and their day-to-day functioning. This retrospective evaluation demonstrates the feasibility and acceptability of Re-PROGRAM as a brief intervention for individuals diagnosed with FS delivered in a clinical outpatient setting and warrants further investigation in larger scale, randomized controlled studies.

Analysis of the Current Situation and the Potential Development of Environmental Education Program Evaluation in Japan: Organizing Research Cases Based on the “Guidelines for Environmental Education Program Evaluation”

Analysis of the Current Situation and the Potential Development of Environmental Education Program Evaluation in Japan: Organizing Research Cases Based on the “Guidelines for Environmental Education Program Evaluation”

Exploring the similarities and differences in the development of environmental education between Japan and Taiwan: Focusing on the orientation of Guidelines for Environmental Education Program Evaluation

Exploring the similarities and differences in the development of environmental education between Japan and Taiwan: Focusing on the orientation of Guidelines for Environmental Education Program Evaluation

Program Evaluation of Pharmacist-Performed Medication Adherence Assessments in Candidates for Living Donor Kidney Transplant.

Introduction: Medication education and adherence assessments are integral to kidney transplant success. This program evaluation aimed to describe candidate-reported findings using a standardized medication adherence assessment in candidates undergoing living-donor kidney transplantation. Design: This was a single-center retrospective description of medication adherence on adult HIV-negative living-donor candidates from July 1, 2018 to December 1, 2018 who had ≥6 months post-operative follow-up. Medication adherence assessments were performed by a pharmacist at the pre-operative visit within 2 weeks prior to transplant. Candidates were considered to (a) have adherence concerns if they reported missed/late medications within 2 weeks of assessment or ever stopped a medication without medical advice and (b) considered using adherence strategies if they reported active use of pill box, method to keep track of refills/auto-refill use, medication list, or medication reminder(s). Missed medication data were collected at 3- and 6-months posttransplant. Results: Among 181 candidates included, 81 (45%) had adherence concerns and 169 (93%) reported using adherence strategies. There were no significant differences with adherence concerns by age ≤ 29 years, sex, race, prior transplant/dialysis, or less than a high school education. More candidates with greater than a high school education used adherence strategies (96% vs 86%, P = .002). Too few candidates had documentation on missing medications at 3 and 6 months. Conclusions: Over 40% of candidates reported characteristics concerning medication nonadherence despite over 90% reporting adherence strategies used. Medication adherence assessments can assist with identification of medication nonadherence and education individualization.

A valid and reliable explanatory model of learning processes in heritage education

BackgroundThe main challenge in heritage education is to identify the verbs—and their hierarchical relations—that explain heritage learning as based on empirical evidence. The Heritage Learning Sequence (HLS) selects seven verbs (Knowing-Understanding-Respecting-Valuing-Caring-Enjoying-Transmitting) on the basis of (a) theoretical studies, (b) analyses of international standards, and (c) evaluation of heritage education programs. The study has the following objectives: (a) to clarify the heritage learning process; (b) to test a theoretical model that groups the verbs that make up the Heritage Learning Sequence (HLS), as well as the relationships between them; (c) to identify possible sub-models that explain the different heritage learning itineraries.MethodsThe Q-Herilearn scale (previously calibrated using SEM and IRT models) was administered to N=1454\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$N = 1454$$\\end{document} individuals, focusing on seven factors (corresponding to each HLS verb) that measure heritage learning. Partial Least Squares Structural Equation Modeling (PLS-SEM) was used as a general analytical strategy.FindingsThe results obtained provided sufficient guarantees to validate the HLS and showed the adequate explanatory and predictive power and general fit of the proposed model (Heritage Learning Model); all twelve hypothesized direct influence relations between the main verbs that define heritage learning were confirmed. The statistical significance values suggested the existence of other internal subsequences that could be explored in further studies.ContributionLearning modeling provides a key structural framework for (a) the design of effective, efficient, and comprehensive tools to measure heritage learning and (b) their operationalization in heritage education designs.

Use and impact of virtual resources for peer support by young adults living with cancer: A systematic review

Background: Young adults (YA) with cancer often have unmet psychosocial needs. The impact of peer support delivered in an online format to fulfil these needs in the populations of YA with cancer has not been thoroughly examined. Methods: We searched Cochrane Central Register of Controlled Trials, CINAHL, Embase, and Medline databases. We included articles about online peer support interventions for YA cancer survivors between the ages of 18 to 40 years old. Results: Our literature search yielded n = 2,773 articles and we obtained consensus on 12 articles for inclusion. We qualitatively synthesized these articles using data abstraction based on the Template for Intervention and Replication (TIDierR) checklist. Overall, six studies demonstrated correlation between online peer support and improved wellbeing of participants. Thus, online peer support may be useful for young cancer patients. Conclusion: This systematic review summarizes the current state of knowledge regarding the availability and evaluation of online YA peer support programs and reveals the need for further research in this field. Keywords: young adult, social support, cancer survivor, social survivorship, social media, internet-based intervention

Evaluation of a medical student-delivered smoking prevention program utilizing a face-aging mobile app for secondary schools in Germany: The Education Against Tobacco cluster-randomized controlled trial

BackgroundTo reduce smoking uptake in adolescents, the medical students' network Education Against Tobacco (EAT) has developed a school-based intervention involving a face-aging mobile app (Smokerface). MethodsA two-arm cluster-randomized controlled trial was conducted, evaluating the 2016 EAT intervention, which employed the mobile app Smokerface and which was delivered by medical students. Schools were randomized to intervention or control group. Surveys were conducted at baseline (pre-intervention) and at 9, 16, and 24 months post-intervention via paper & pencil questionnaires. The primary outcome was the difference in within-group changes in smoking prevalence between intervention and control group at 24 months. ResultsOverall, 144 German secondary schools comprising 11,286 pupils participated in the baseline survey, of which 100 schools participated in the baseline and at least one of the follow-up surveys, yielding 7437 pupils in the analysis sample. After 24 months, smoking prevalence was numerically lower in the intervention group compared to control group (12.9 % vs. 14.3 %); however, between-group differences in change in smoking prevalence between baseline and 24-months follow-up (OR=0.83, 95 %-CI: 0.64–1.09) were not statistically significant (p = 0.176). Intention to start smoking among baseline non-smokers declined non-significantly in the intervention group (p = 0.064), and remained essentially unchanged in the control group, but between-group differences in changes at the 24-months follow-up (OR=0.88, 0.64–1.21) were not statistically significant (p = 0.417). ConclusionWhile a trend towards beneficial effects of the intervention regarding smoking prevalence as well as intention to start smoking among baseline non-smokers was observed, our smoking prevention trial demonstrated no significant effect of the intervention.

Ageism toward older adults among social education students: the impact of a photovoice program

ABSTRACT This study describes the design, implementation, and evaluation of a photovoice program for Social Education students from a Polytechnic University in Portugal. The program was named ‘Ageism in Perspective’ and developed over six sessions during one semester. The design was based on principles of the original photovoice methodology, along with previous experiences of photovoice as an active and experiential learning methodology in higher education. Implementation included theoretical-pedagogical and ethical activities about the methodology (photovoice) and the topic under exploration (ageism). It went practically-technologically through the process of taking and presenting photographs in the group/classes, ending with a seminar and exhibition open to the community to share the main findings. Evaluation included a pre-posttest approach using the Negative Stereotypes Toward Aging Questionnaire (CENVE) and a final focus group addressing students’ overall experience with the program, the photovoice approach, and the impact on their perspectives about ageism. Participants (n = 45; mean age of 21.1, SD 4.1) were students of the second year in the degree course of Social Education. There was a statistically significant decrease in the CENVE score between the pre (30.1 ± 7.3) and the post (21.5 ± 4.0) test, meaning lower stereotypes after the program. Emerging themes from the focus group thematic analysis were (i) I am more willing to interact with older adults; (ii) Photovoice booster multiple perspectives and reflexivity; (iii) Becoming aware that ageism exists (even in myself). Findings suggest that photovoice is a good pedagogical tool to address students’ stereotypes about older adults, which should be as important as addressing the syllabus content.

Study protocol for a single-site feasibility study evaluating the adoption and fidelity of Prep-4-RT: prehabilitation for head and neck cancer patients undergoing radiotherapy

IntroductionPrep-4-RT is a co-designed stepped-care multimodal prehabilitation program for people scheduled to receive radiotherapy for head and neck cancer (HNC). Prehabilitation, which occurs between diagnosis and treatment commencement, aims to improve a patient’s health to reduce the incidence and severity of current and future impairments. HNC treatment can be distressing and has detrimental impacts on function and quality of life. HNC patients have increased social vulnerabilities including higher rates of socio-economic disadvantage and engagement in lifestyle habits which increase cancer risk. High levels of physical and psychological impacts of HNC treatment and increased social vulnerabilities of this population warrant investigation of optimal pathways of care, such as prehabilitation. This paper describes a research protocol to evaluate the feasibility of Prep-4-RT, which was designed to prepare HNC patients for the physical and psychological impacts of radiotherapy.Methods and analysisAt least sixty adult HNC patients, scheduled to receive radiotherapy (with or without chemotherapy), will be recruited over a five-month period. All participants will receive access to Prep-4-RT self-management resources. Participants identified through screening as high-risk will also be offered individualised interventions with relevant allied health professionals prior to the commencement of radiotherapy (psychologists, dietitians, speech pathologists and physiotherapists). Participants will complete evaluation surveys assessing their experiences with Prep-4-RT resources and interventions. Clinicians will also complete program evaluation surveys. Primary feasibility outcomes include adoption (uptake and intention to try) and fidelity (adherence to the specialist prehabilitation pathway). Secondary feasibility outcomes include acceptability (patient and clinician) of and satisfaction (patient) with Prep-4-RT as well as operational costs. Feasibility outcome data will be analysed using exact binomial and one-sample t tests, as appropriate.Ethics and disseminationEthics approval has been obtained at the Peter MacCallum Cancer Centre in Melbourne, Australia. Results will be presented at national conferences and published in peer-reviewed journal(s) so that it can be accessed by clinicians involved in the care of HNC patients receiving radiotherapy. If the model of care is found to be feasible and acceptable, the transferability and scalability to other cancer centres, or for other cancer types, may be investigated.Registration detailsANZCTA (Australian New Zealand Clinical Trials Registry) ACTRN12623000770662.

Evaluation Resources for Asthma Programs and Beyond.

Evaluation can ensure the quality of public health programs. Systematic efforts to identify and fully engage everyone involved with or affected by a program can provide critical information about asthma programs and the broader environment in which they operate. To assist evaluators working at programs funded by the Centers for Disease Control and Prevention (CDC's) National Asthma Control Program (NACP), we developed a package of tools that build on the CDC's 1999 Framework for Program Evaluation in Public Health. The resulting suite of evaluation tools guides evaluators through a structured but flexible process, engaging a diverse array of interest holders and actively involving them in evaluation planning and implementation, all while strengthening their capacity to meaningfully contribute to the evaluation process. For our newest tool, our team reviewed the recent evaluation literature to create an enhanced version of the 1999 framework that describes important elements of professional evaluation practice. Although the original framework describes the steps to take in conducting an evaluation and the standards for a high-quality evaluation, our enhanced framework includes an explanation of how evaluators should approach their work: by incorporating critical reflection, interpersonal competence, situational awareness, and cultural responsiveness. In this article, we highlight many of the evaluation resources our team has created since the NACP's inception, culminating in a free e-text called Planting the Seeds of High-Quality Program Evaluation in Public Health. Public health professionals working in many types of programs - not just asthma - may find these resources useful.

Impact of Implementation Facilitation on the REACH VET Clinical Program for Veterans at Risk for Suicide.

In 2017, the Veterans Health Administration (VHA) implemented a national suicide prevention program, called Recovery Engagement and Coordination for Health-Veterans Enhanced Treatment (REACH VET), that uses a predictive algorithm to identify, attempt to reach, assess, and care for patients at the highest risk for suicide. The authors aimed to evaluate whether facilitation enhanced implementation of REACH VET at VHA facilities not meeting target completion rates. In this hybrid effectiveness-implementation type 2 program evaluation, a quasi-experimental pre-post design was used to assess changes in implementation outcome measures evaluated 6 months before and 6 months after onset of facilitation of REACH VET implementation at 23 VHA facilities. Measures included percentages of patients with documented coordinator and provider acknowledgment of receipt, care evaluation, and outreach attempt. Generalized estimating equations were used to compare differences in REACH VET outcome measures before and after facilitation. Qualitative interviews were conducted with personnel and were explored via template analysis. Time had a significant effect in all outcomes models (p<0.001). An effect of facilitation was significant only for the outcome of attempted outreach. Patients identified by REACH VET had significantly higher odds of having a documented outreach attempt after facilitation of REACH VET implementation, compared with before facilitation. Site personnel felt supported and reported that the external facilitators were helpful and responsive. Facilitation of REACH VET implementation was associated with an improvement in outreach attempts to veterans identified as being at increased risk for suicide. Outreach is critical for engaging veterans in care.

An In-Home Withdrawal Service for individuals with low-to-moderate substance dependence: implementation and program evaluation.

Background Sonder's In-Home Withdrawal Service (IHWS) has been providing a unique home-based, multidisciplinary, wraparound withdrawal option for people with low-to-moderate levels of substance dependence since September 2019. This paper provides an evaluation of the service's overall impact on reducing substance use among clients through the delivery of this innovative service model. The evaluation explores the acceptability and feasibility of the service via client, stakeholder, and staff perspectives. Methods A mixed-method design was used to evaluate the IHWS. Descriptive analysis of quantitative data was conducted using clinical assessments from client records and online feedback surveys. Qualitative data from client, staff, and stakeholder feedback surveys were analysed thematically. Results Overall, 1166 referrals were received over the lifespan of the service, and a data set of 96 clients was included in the analysis. Self-reported measures showed that most clients decreased their substance use (89%), improved their psychological health status (75%), improved their physical health (65%), improved their quality of life (69%), and improved their understanding and ability to manage their alcohol and other drugs (AOD) use (84%). Client feedback suggests the service is providing a unique option for AOD withdrawal. Stakeholders commend the service's home-based setting, multidisciplinary and person-centred approach to care, and recommended expansion of the service to increase access for clients and reduce demand on inpatient settings. Conclusions The IHWS is having a significant impact in reducing substance use and highlights the need for increased access to holistic approaches to withdrawal. This includes pre- and post-withdrawal support and the inclusion of multidisciplinary teams, and engaging lived experience practitioners. A focus on funding primary-based services is required to meet the rising costs of tertiary-based care and to better meet the needs of consumers.

Formative qualitative evaluation of an improvement programme delivered in an English hospital trust to reduce harm from pressure ulcers

BackgroundPressure ulcers (PUs) are a leading cause of preventable harm globally and can cause patients significant pain, infection and, in rare incidents, death. There is a strong evidence base for...