Results Of Program Evaluation Research Articles

A Monitoring Tool for the Evaluation of Program Outcomes in Engineering Education

ABSTRACTDuring the evaluation process, organizations that accredit engineering programs determine whether a program meets a list of predetermined criteria. One criterion on the list is Program Outcomes (POs). The most common weakness statement is found in this criterion. First, we aim to develop a model that can support program evaluation during the accreditation process. Second, the proposed approach can be used as a feedback tool in the continuous improvement of teaching in engineering programs. The course instructor runs the first module, which predefined learning outcomes and the POs are linked together using an Excel‐based software for each activity performed in the course. The second module calculates the scores that reflect the achievement levels of the POs for the whole program. The scores from individual courses are simultaneously transferred to the Technique for Order Preference by Similarity to Ideal Solution module that determines the rating scores of the POs in the program. The proposed system shows which POs are at a satisfactory level and which are lagging behind and need attention. It helps the course administrators to manage the course material proactively. The proposed system is a powerful tool for monitoring and tracking POs, and it is the primary tool used by program administrators to improve the program continuously. It is particularly useful in identifying the weak spots of the program. The successful implementation of the system in our department since 2012 makes us confident that the system could be adopted by other engineering programs.

Pola Penerapan Residensi Unggulan Melalui Pendekatan In-kind dan Matchmaking dalam Pengembangan Media Pembelajaran Seni Lintas Disiplin bersama Industri Kreatif Program Studi Pendidikan Seni Rupa

The implementation pattern of flagship residencies through in-kind and matchmaking approaches in developing interdisciplinary art learning media with the creative industry at the Fine Arts Education Study Program aims to enhance the quality and relevance of art education. This program integrates in-kind contributions from the creative industry with a matchmaking process to create productive collaborations between students and industry practitioners. Methods include residency selection, interdisciplinary art project development, and evaluation of program outcomes and impacts. The results show an increase in students' practical skills and interdisciplinary understanding, as well as the enrichment of the fine arts education curriculum through direct input from the creative industry. The implementation of this pattern is expected to strengthen the synergy between education and industry, and produce innovative and applicable art learning media

Effects of a Simulation-Based Care After-Death Mentoring Program for New Nurses: Augmented Reality End-of-Life Experience.

Background: Nurses who provide person-centered care even after death must possess essential competencies to perform their duties effectively. Therefore, it is imperative to develop educational programs to enhance the capabilities of new nurses in care after death. Objective: To develop a care after-death mentoring program that includes an augmented reality (AR) end-of-life experience for new nurses and to describe its effectiveness. Design: A quasi-experimental pre- to post-test design was followed to evaluate program outcomes. Setting/Participants: The participants (n = 18) were nurses with <12 months of experience working at a tertiary general hospital in South Korea. Measurements: A pre-post survey was conducted on comfort in bereavement/end-of-life care, death anxiety, and compassion competency. Data were analyzed using the SPSS program, employing descriptive statistics and paired sample t-tests. Results: Significant improvements were observed in both comfort during bereavement/end-of-life careand compassion competency (t = -8.43, p < 0.001; t = -4.90, p < 0.001). Conclusions: This study demonstrated enhancements in participants' comfort levels regarding bereavement and end-of-life care, as well as their ability to exhibit compassion after participating in the program. Consequently, it was confirmed that simulation-based care after-death mentoring education utilizing an AR app helped enhance the capabilities of new nurses.

Collaborations to Support the SJ4A Curriculum in Archival Studies and Social Justice Intersections: Training of Community-Embedded Paraprofessional Archivists Who Are Black, Indigenous, and People of Color

The Institute of Museum and Library Services’ Laura Bush 21st Century Librarian Program recently awarded a $463,238 grant titled “Training of Community-Embedded Social Justice Archivists” (SJ4A) to the School of Library and Information Studies atthe University of Alabama. The grant’s goal is to recruit/train 17 community-embedded paraprofessional archivists who are Black, Indigenous, and people of color to complete their master’s degree part-time from January 2023 – December 2024 in the SLIS’ synchronous distance education program. SJ4A meets an urgent need to diversify the professions and for archivists to apply social justice competencies in work experience and community-embedded practice with graduate instruction and curriculum support. This paper identifies ongoing collaborations to support curriculum development in the SJ4A phases, including recruitment of students,needs assessment of archives-related professionals about social justice services, implementation of educational activities, mentoring, evaluation of program outcomes, dissemination of results, and post-graduation career tracking.

Residential treatment program outcome evaluation: one agency’s discoveries about performance measurement

ABSTRACT This paper explores the design and implementation of a comprehensive, continuous outcome evaluation for youth in therapeutic residential care (TRC) to measure program efficacy, based on The Joint Commission (TJC) accreditation guidelines. Over a 10-year implementation period, researchers analyzed changes in 281 residential care youth from baseline to reassessment after 4–6 months in treatment. The primary assessment instruments included the SASSI-A2, the FACES III, the Measure of Psychosocial Development (MPD), the Behavioral and Emotional Rating Scale (BERS), and the Youth Comprehensive Risk Assessment (YCRA). Results indicated reductions in the risk areas of self-harm, conduct disorder behaviors, and substance abuse. Significant increases occurred in psychosocial development, family engagement, and personal strengths. However, results also revealed continued low levels of youth development in trust vs mistrust, ego integrity vs despair, willingness to disclose loss of control, recognition of negative consequences of substance misuse, and response to conduct disorder treatments. Implications and recommendations for ongoing program evaluation, staff development, and institutional change are discussed.

Acceptability and feasibility of a group-based intervention to improve outcomes for children at risk for developmental delays in Kenya: A piloted randomized trial.

The Care for Child Development (CCD) program may improve child development outcomes in resource-limited settings, but has not yet been adapted to group-based settings to facilitate sustainable dissemination. In this study, we determined the acceptability and feasibility of a group-based CCD program, with evaluation of program outcomes for child development, home environment, and symptoms of maternal depression as secondary outcomes. We evaluated this adapted program using a 2 × 2 crossover-designed pilot study administered over 10 bi-weekly sessions. Acceptability and feasibility were assessed through focus group discussions using qualitative methods. Child development, home observations, and symptoms of maternal depression were evaluated at baseline, 6months, and 12months and assessed quantitatively. Twenty-six mother-child dyads participated. Overall, they perceived CCD as acceptable and feasible, and especially beneficial within its group-based format. Although there were no measured improvements in child development, improvements in stimulating home environments (mean difference 2.5, 95% C.I. [0.37, 4.72]) were found. Further scale-up of this intervention is needed to determine effectiveness.

PROGRAM RUMAH CERDAS DIGITAL SEBAGAI UPAYA PENINGKATAN LITERASI DIGITAL DI DESA MALLONGI-MALLONGI

Digital literacy can be defined as the ability to use and understand information from a very wide range of sources that can be displayed on computer devices. The measurement of the digital literacy index is seen from four dimensions, namely, digital skills, digital ethics, digital security, and digital culture. When measured against these four dimensions, the level of digital literacy, Mallongi-Longi Village is considered as low in its application. Therefore, the Student Activity Unit (PPK Ormawa) of the Mathematics Department at FMIPA UNM seeks to address the issues in Mallongi-Longi Village by implementing a work program called the Digital Smart Home Program. It is hoped that this Smart Home Program can broaden the knowledge, attitudes, and skills of the Mallongi-Lonngi Village community regarding digital literacy. This program is conducted through classes and seminars. It consists of two main activities, namely digital classes and digital workshops, targeting the Mallongi-Longi Village community. This program runs for 58 days. Based on the program evaluation results, it was found that the target participants have responded well, the target community wants the sustainability of this program and this program has achieved all the objectives of the program implementation.

The Johns Hopkins Physician-Scientist Training Program to Enhance Institutional Retention and Entry Into Academic Positions: An Evaluation of Program Effectiveness and Outcomes.

The Johns Hopkins Physician-Scientist Training Program (PSTP) was implemented to overcome well-documented challenges in training and retaining physician-scientists by providing physician-scientist pathway training for residents and clinical fellows. The program's core tenets include monthly seminars, individualized feedback on project proposals, access to mentors, and institutional funding opportunities. This study evaluated the effectiveness and outcomes of the PTSP and provides a framework for replication. A query of institutional demographic data and bibliometric variables of the PSTP participants (2017-2020) at a single academic medical center was conducted in 2021. In addition, a voluntary survey collected personal and program evaluation information. Of 145 PSTP scholars, 59 (41%) were women, and 41 (31%), 8 (6%), and 6 (5%) of scholars self-identified as Asian, Hispanic, and Black, respectively. Thirty-three (23%) scholars received PSTP research support or career development microgrants. Of 66 PSTP graduates, 29 (44%) remained at Johns Hopkins as clinical fellows or faculty. Of 48 PSTP graduates in a post-training position, 42 (88%) were in academia, with the majority, 29 (76%), holding the rank of assistant professor. Fifty-nine of 140 available participants responded to the survey (42% response rate). The top-cited reason for joining the PSTP was exposure to mentors and administration (50/58 respondents, 86%), followed by seeking scholarly opportunities (37/58 respondents, 64%). Most scholars intended to continue a career as a physician-scientist. The PSTP provides internal research support and institutional oversight. Although establishing close mentor-mentee relationships requires individualized approaches, the PSTP provided structured academic pathways that enhanced participating scholars' ability to apply for grants and jobs. The vast majority continued their careers as physician-scientists after training. In light of the national evidence of a "leaky physician-scientist pipeline," programs such as the PSTP can be critical to entry into early academic career positions and institutional retention.

Consensus Guidelines for Assessment and Comprehensive Outcomes Evaluation in Rural Pain Clinics

Recent advancements in nurse anesthesiology fellowship training programs have helped to establish advanced pain management services for rural communities. Consensus guidelines to direct the rural provider toward the most valid and reliable measures for pain assessment and functional outcomes evaluation are not presently available. The primary aim of this initiative was to establish consensus guidelines for a comprehensive outcome evaluation program with specific time intervals for assessments that can be utilized by all rural pain clinics. The American Association of Nurse Anesthesiology Nonsurgical Pain Management Advisory Panel members provided formative and expert feedback for this initiative. The Delphi model was utilized to achieve consensus through multiple rounds of surveys. Items achieving >70% agree/strongly agree were kept; items with >70% disagree/strongly disagree were rejected; items meeting neither advanced to the following round for evaluation until consensus was met. During round I, consensus was reached for: (1) the use of the Numerical Rating Scale for pain severity; and (2) timing of pain severity & functional pain outcomes on each office visit and before/after each intervention. Round II, consensus was achieved for: (1) the use of Wong-Baker FACES Pain Rating Scale as a suitable instrument when literacy or communication pose a barrier but not as a primary assessment; and (2) the use of the Brief Pain Index-Short Form for functional outcome measures. During round III, consensus was reached for: (1) the use of the Oswestry Disability Index as a functional outcome measure; and (2) pain reassessment being performed within 14 days of intervention. This initiative provides rural pain clinics with a comprehensive outcome evaluation program with specific time intervals for assessments.

Implementation and Evaluation of a Large Community-Based Colorectal Cancer Screening Program.

Colorectal cancer (CRC) screening can significantly reduce incidence and mortality; however, screening rates are suboptimal. The lowest rates are among those with no usual source of care and the uninsured. We describe the implementation and evaluation of a community-based CRC screening program from 2012 to 2015 designed to increase screening within a predominantly Hispanic US-Mexico border population. The multicomponent, evidence-based program provided in-person, bilingual, culturally tailored health education facilitated by community health workers, no-cost primarily stool-based testing and diagnostic colonoscopy, and navigation. We recruited uninsured individuals due for CRC screening from clinics and community sites. An extensive qualitative and quantitative program process and outcome evaluation was conducted. In total, 20 118 individuals were approached, 8361 were eligible for screening; 74.8% completed screening and 74.6% completed diagnostic testing; 14 cancers were diagnosed. The mean age of participants was 56.8 years, and the majority were Hispanic, female, and of low socioeconomic status. The process evaluation gathered information that enabled effective program implementation and demonstrated effective staff training, compliance with processes, and high patient satisfaction. This program used a population-based approach focusing on uninsured individuals and proved successful at achieving high fecal immunochemical test kit return rates and colonoscopy completion rates. Key factors related to its success included tailoring the intervention to our priority population, strong partnerships with community-based sites and clinics, expertise in clinical CRC screening, and an active community advisory board. This program can serve as a model for similar populations along the border to increase CRC screening rates among the underserved.

Adopting the RE-AIM analytic framework for rural program evaluation: experiences from the Advance Care Planning via Group Visits (ACP-GV) national evaluation.

To support rigorous evaluation across a national portfolio of grants, the United States Department of Veterans Affairs (VA) Office of Rural Health (ORH) adopted an analytic framework to guide their grantees' evaluation of initiatives that reach rural veterans and to standardize the reporting of outcomes and impacts. Advance Care Planning via Group Visits (ACP-GV), one of ORH's Enterprise-Wide Initiatives, also followed the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. ACP-GV is a national patient-centered intervention delivered in a large, veterans integrated healthcare system. This manuscript describes how RE-AIM was used to evaluate this national program and lessons learned from ORH's annual reporting feedback to ACP-GV on their use of the framework to describe evaluation impacts. We used patient, provider, and site-level administrative health care data from the VA Corporate Data Warehouse and national program management databases for federal fiscal years (FY) spanning October 1, 2018-September 30, 2023. Measures included cumulative and past FY metrics developed to assess program impacts. RE-AIM constructs included the following cumulative and annual program evaluation results. ACP-GV reached 54,167 unique veterans, including 19,032 unique rural veterans between FY 2018 to FY 2023. During FY 2023, implementation adherence to the ACP-GV model was noted in 91.7% of program completers, with 55% of these completers reporting a knowledge increase and 14% reporting a substantial knowledge increase (effectiveness). As of FY 2023, 66 ACP-GV sites were active, and 1,556 VA staff were trained in the intervention (adoption). Of the 66 active sites in FY 2023, 27 were sites previously funded by ORH and continued to offer ACP-GV after the conclusion of three years of seed funding (maintenance). Lessons learned developing RE-AIM metrics collaboratively with program developers, implementers, and evaluators allowed for a balance of clinical and scientific input in decision-making, while the ORH annual reporting feedback provided specificity and emphasis for including both cumulative, annual, and rural specific metrics. ACP-GV's use of RE-AIM metrics is a key step towards improving rural veteran health outcomes and describing real world program impacts.

AN OVER VIEW ON EVALUATION OF TRAINING PROGRAM

This paper begins with an introduction highlighting the significance of training programs and the crucial role of effective evaluation in optimizing outcomes. The purpose of training program evaluation is then explored, focusing on identifying goals, aligning objectives with organizational aims, and understanding the impact on individual and organizational performance. The article delves into various types of training program evaluation, including formative, summative, process, and outcome evaluations, along with their respective methods. Key metrics and indicators, both quantitative and qualitative, are discussed, covering completion rates, performance improvement, cost-effectiveness, participant feedback, skill transfer, and changes in organizational culture. The challenges associated with training program evaluation are addressed, such as a lack of clear objectives, difficulty in measuring intangible outcomes, limited resources, and resistance to change. The abstract concludes by highlighting future trends, emphasizing technology integration, ROI measurement, and personalized learning. It wraps up with a recap of the importance of training program evaluation and an encouraging call for organizations to adopt effective practices for continuous improvement. Keywords: Training Program, Evaluation, Optimization, Formative and Summative valuation, Metrics and Indicators

EDUKASI PANGAN DAN JAJANAN YANG AMAN SEHAT SERTA BERMUTU DI SMA NEGERI 20 BATAM

This community service initiative is designed to provide an alternative educational platform targeting both educators and students. Furthermore, it seeks to engage the participation of students' parents and committees in supporting school food safety programs. Concurrently, educators are equipped with the skills to enhance the management of school canteens, ensuring the provision of snacks that meet stringent safety, quality, and nutritional standards. The primary objective is to cultivate awareness among all members of the school community regarding the paramount importance of selecting snacks that are not only safe but also of high quality and nutritional value. It is anticipated that this heightened awareness will empower the community to make judicious choices in their food consumption. The methodology employed in this community service initiative includes the dissemination of learning materials and references through structured presentations and interactive sessions, incorporating a question-and-answer format. Participants undergo a comprehensive learning process, acquiring knowledge and practical skills through demonstrative sessions that are subsequently applied. Through the evaluation of program implementation and outcomes, factors supporting or hindering the success of this community service initiative can be discerned. The socialization process places a particular emphasis on elucidating the potential hazards associated with the misuse of substances in food, such as rhodamine B, methanyl yellow, formalin, and borax. Participants are not only informed about these dangers but are also provided with guidance on effectively educating and guiding students towards the consumption of healthful snacks. The initiative additionally aspires to motivate educators to prioritize the psychological well-being of their students, with the ultimate goal of fostering independence within the school community concerning food safety.

Program evaluation of postgraduate obstetrics and gynecology training in Lao people’s democratic republic - using the CIPP model

BackgroundThe obstetrics and gynaecology (OB-GYN) residency training program in Lao People’s Democratic Republic (PDR) began in 2003 based on the Millennium Development Goals (MDGs) and ‘Reproductive, maternal, newborn, and child health interventions (RMNCH) strategies and action plan’. However, the training program had not been properly evaluated previously. The purpose of this study is to evaluate the current postgraduate OB-GYN residency training program in Lao PDR by using CIPP model to identify the current problems (the strengths and weaknesses) and suggest a future plan to promote continuous improvement.MethodThe context, input, process, and product classification (CIPP) model was used to develop criteria and indicators. A mixed-methods approach was used for this study. To capture instructional material for quantitative analysis, a Google survey with 38 items and a t-test were used to determine a significant difference in responses between residents and lecturers (N = 120). Based on qualitative analysis, an in-depth interview was done (four questions based on study outcomes, including satisfaction, strengths and weaknesses, and future opportunities), and six interviews provided different viewpoints on the course. The SPSS software program was used to measure validity, with p-values = 0.05.ResultsThe overall average response rate was 97.5%. Two significant differences in program perspectives were revealed between lecturers and residents, difficulties in maintaining the course (professors 3.66 ± 1.03 and residents 3.27 ± 0.98, p = 0.04) and learning outcomes achieved (professors 3.57 ± 0.85 and residents 3.14 ± 0.95, p = 0.01 The overall average for the context part of the questionnaire was under 3.00, with the lowest scores for overlapped learning outcomes and difficulties in maintaining the course. The input part, lack of the classroom, skills lab and staff; the process part, lecturer to collect student opinions and the product part on learning outcomes.ConclusionCurriculum improvement based on the program evaluation results, including regular evaluation and feedback, will advance the residency training program based on the RMNCH strategy and contribute to the promotion of maternal health in the Lao PDR.

Building Resilience through Intervention, Direct Guidance, and Empowerment (BRIDGE): A demonstration project to assist youth aging out of foster care in Poland

Building Resilience through Intervention, Direct Guidance, and Empowerment (BRIDGE) is an innovative program designed to prevent homelessness and other negative outcomes among youth aging out of foster care. BRIDGE was pilot-tested on youth aging out of two orphanages in a city in southern Poland in 2009–2012. Youth were recruited at age 17, before aging out at age 18. Quasi-experimental methods were used to evaluate program outcomes among nine BRIDGE clients compared to two matched groups (n = 9 and n = 18) of youth who had recently aged out of orphanages in the same Polish region. Consistent with Hypothesis 1, results of chi-square tests showed that, over an 18-month follow-up period, BRIDGE clients experienced superior housing outcomes relative to both matched groups. Consistent with Hypothesis 2, the results of a between-subjects ANOVA showed that BRIDGE clients received significantly greater income relative to the matched groups. Within-subjects effects of BRIDGE from baseline across 6-, 12-, and 18-month follow-ups were examined with repeated-measures ANOVAs. Consistent with Hypotheses 3–5, results showed that across time BRIDGE clients experienced a general increase in income and decrease in psychological distress (i.e., total distress, obsessive-compulsiveness, interpersonal sensitivity, and paranoia). The study’s findings support the further development of BRIDGE and similar programs.

Implementation of a mobile prosthetic and orthotic care program in the VA; a qualitative study of implementation challenges and associated strategies for improvement.

Anticipating and addressing implementation challenges is critical to ensuring success of mobile healthcare programs. Mobile Prosthetic and Orthotic (O&P) Care (MoPOC) is a new U.S. Department of Veterans Affairs (VA) program that aims to improve access to VA-based O&P services through a national network of traveling O&P clinicians who deliver care in rural communities. We conducted an iterative evaluation guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to identify challenges and associated strategies for successful implementation of this mobile O&P program. MoPOC is delivered by an O&P clinician anchored at a VA medical center (VAMC). Clinicians travel to remote VA clinics and Veteran's homes with a custom vehicle which provides storage and a workshop to modify O&P devices. Each clinician is supported by a program support assistant. MoPOC was implemented in three phases. The qualitative evaluation of MoPOC implementation was conducted as part of a larger evaluation of MoPOC program outcomes. We conducted semi-structured interviews and regular check-ins with MoPOC clinicians, site managers, and stakeholders both prior to implementation and throughout the implementation process. Interviews were recorded and transcribed verbatim. Data was analyzed across sites and comparatively by phase using a rapid matrix analysis to identify themes related to adoption and implementation challenges and key strategies developed to address those challenges. We identified four key themes related to successful program implementation, each with associated challenges and improvement strategies: (1) "Finding the right sites for MoPOC" through intentional recruitment and site selection; (2) Identifying the "sweet spot": Balancing program capacity, sustainability, and MoPOC clinician satisfaction; (3) Shifting from testing to standardizing; and (4) "Being strategic with hiring" to improve program adoption. Implementation challenges were related to recruiting and selecting successful sites, ensuring timely program adoption, balancing site level adaptation and program standardization, and scaling programs to enhance efficiency, reach, and satisfaction. An iterative approach guided by the RE-AIM framework resulted in program improvement and more rapid implementation in each successive phase. The challenges described in MoPOC implementation may be common issues in implementing new mobile programs in rural areas.

CREATING STRUCTURES FOR SUSTAINABILITY OF EVIDENCE-BASED DEMENTIA CAREGIVER PROGRAMS

Abstract Ensuring sustainable impact for broad-based evidence-based programs (EBPs) serving caregivers for persons living with dementia in the U.S. requires systemic maintenance of programmatic fidelity. This symposium aims to assist decision makers, aging and public health networks, and intervention developers to anticipate and deploy effective strategies and resources required to implement EBP interventions and sustain them. A series of four papers describes precursor studies and activities leading to the development of a structure aimed at ensuring that one such EBP, the Savvy Caregiver® (Savvy), can be faithfully sustained by sponsoring organizations. The first paper presents results from in-depth qualitative interviews of key staff (n=25) in nine organizations delivering Savvy. Findings highlight organizational, structural, and regional collaborations that contributed to sustained EBP implementation with fidelity. The second paper describes an online program to train Savvy interventionists. Informed by the first paper’s findings a fully asynchronous multi-media professional continuing educational interventionist training program was developed in collaboration with the educational design team at Emory’s School of Nursing. The third paper presents results of the online trainer program evaluation conducted among 76 interventionists from 32 organizations in eight states. High scores on content comprehension (95-99%) and participant feedback document benefits of self-paced learning, interactive materials and quizzes, and self-reflective exercises. The final paper describes a platform (Savvy Systems, LLC) created to ensure Savvy’s long-term dissemination, programmatic integrity and sustained use. The single platform provides for authorized content dissemination, leader training, enhanced referrals, and copyright compliance through licensing. Discussant will offer policy and practice implications. This is a collaborative symposium between the Alzheimer’s Disease and Related Dementias and Family Caregiving Interest Groups.

Bridging Thai music notation to Western music scores through innovative conversion and evaluation

Research on bridging Thai music notation to Western music scores through innovative conversion and evaluation by developing a Thai music notation printing program through innovative research For converting Thai musical notation into an international musical score by researching documents and related research along with advice from music experts. After that, the program was tried and evaluated by three experts and 20 test users. The results of the research showed the creation of a program to Thai music notation to Western music note structure conversion called XxX Notation (Triple Note) by accessing the domain http://www.music.msu.ac.th/xxxnotation-v2, which is under the website of the College of Music, Mahasarakham University. Music can be saved online, reused, and edited. Able to export notes to paper through printers, PDF, Word, and Excel. Able to listen to the sound of printed notes. It is the sound of a large gong, which is an instrument in the main melody of Thai music. And can increase or decrease speed. To Thai music notation to Western music note structure conversion. Use Thai music notation input, convert the data to MusicXML, and display it as an international note. The results of the quality assessment of the Thai music notation printing program from 3 experts to evaluate the quality of the program in 4 aspects found that the program evaluation results were the highest quality with an average of 4.68, and the quality assessment results from 20 users were the highest quality with an average of 4.76.

Remedial education program for Syrian refugees: ensuring their learning during a protracted crisis

PurposeThe purpose of this paper is to explore how vulnerable refugee children's education can be supported in the first asylum country during a long-term, complex crisis. More specifically, the authors examine the impact of a remedial education (RE) program on academically challenged Syrian refugees' sustained learning and well-being in Jordan during a protracted emergency.Design/methodology/approachUsing the 2012 United Nations human security framework, the authors analyze the quantitative results of program evaluation, supplemented by qualitative surveys and stakeholder interviews that asked about the aspirations of refugee children and their guardians, their school experience and the refugee–host community relationship.FindingsThe authors' data suggest that the program enhanced targeted Syrians' protection and empowerment. Their increased sense of safety and improved academic performance and learning motivation were likely owing to child-friendly classroom management and pedagogies that facilitated interactive and differentiated learning. Their human security was further strengthened when they learned with Jordanian children who experienced similar academic challenges and with teachers sympathetic to their plight. Meanwhile, a human security framework calls for humanitarian agencies' strategic engagement with local partners to ensure refugees' learning continuity.Originality/valueRefugee education studies in first asylum countries are rare. Even rarer are studies focusing on academically low-achieving refugees with full consideration of the protracted and complex nature of a crisis. This study suggests effective measures to increase their human security while calling for humanitarian aid organizations to use long-term thinking.

Implementation of an Adapted Fidelity Scale for Pennsylvania Coordinated Specialty Care Programs for First Episode Psychosis.

Coordinated Specialty Care (CSC) is an evidence-based model for early intervention of first episode psychosis (FEP). Monitoring fidelity to CSC models is essential for proper evaluation of program outcomes and quality improvement. To address variability across CSC programs and fidelity assessment-associated burden, an adapted fidelity scale was developed and implemented statewide in Pennsylvania. This report describes the design and preliminary performance of the adapted scale. The Pennsylvania FEP Fidelity Scale (PA-FEP-FS) assesses adherence to the CSC model by focusing on essential model components of other established fidelity scales, in tandem with program evaluation data. Initial data from fourteen PA-FEP sites from 2018 to 2021 were examined as preliminary validation. Assessment-associated burdens and costs were also estimated. PA-FEP-FS captured essential components of CSC models and proved feasible for implementation across varying program structures, while minimizing burden and cost. Programs utilized annual feedback as CSC benchmarks, evidenced by increased scores over time. PA-FEP-FS provides a model for adapting CSC fidelity scales to meet state- or local-level requirements while reducing assessment burdens and costs that may be associated with existing scales.