Program Evaluation System Research Articles

Cascade of care in people living with HIV in Iran in 2019; how far to reach UNAIDS/WHO targets

ABSTRACT Systematic HIV program evaluation requires looking at all steps of the HIV cascade of care, from diagnosis to treatment outcomes. Our study was carried out to assess the treatment cascade of people living with HIV (PLWH) in Iran in 2019. We used data from the HIV Case Registry System of Iran through December 2019. We estimated the number of PLWH in 2019 by using Spectrum, and then the proportion of them being diagnosed, linked to care, received antiretroviral treatment and suppressed viral load. We estimated that there are 59,314 (UI: 32,685–125,636) PLWH in Iran, of whom 22,054 people (37% of PLWH) were diagnosed. At the end of 2019, of whom, 14,685 (25% of PLWH) people received antiretroviral therapy. Also, of whom 6338 (11% of PLWH) people had viral load suppression by 2019. Our results showed that about one-third of total PLWH were diagnosed, while this defect is somewhat less in children than adults. To reach the 90.90.90 targets Iran needs to developed the current national HIV care guidelines, which recommend best strategies to scale up the case finding and linkage to care among undiagnosed people specifically those who infected by sexual contact in general and key populations as well.

Hemşirelik Lisans Eğitim Programının Bağlam, Girdi, Süreç ve Ürün (CIPP) Modeli Kullanılarak Değerlendirilmesi

The aim of this study is to evaluate the undergraduate curriculum of a nursing program in accordance with nursing students' and instructors' opinions by using the Context, Input, Process, and Product (CIPP) model. This is a descriptive study which was conducted between September 2017 and July 2018 with 448 students and 82 instructors of a faculty of nursing. The Student and Instructor Information Form and Nursing Undergraduate Curriculum Evaluation Form (NUCEF) were used in order to collect data. The NUCEF consists of 50 items under four sub-dimensions called the Context, Input, Process and Product. The obtained data were analyzed using the SPSS 20.0 program. The frequency, percentage, mean, and standard deviation tests were used in the data analysis. The level of satisfaction with the curriculum was found to be 4.47±2.09 for the students and 6.80±1.89 for the instructors. 42% of the students stated that they thought the program outcomes were not achieved, and 42.7% of the instructors stated they thought the program outcomes were partially achieved. When the distribution of responses of Context, Input, Process and Product sub-dimensions of the NUCEF was examined; the mean item score of the students was found to be between 2.27±1.15 and 3.83±1.06, and the mean item score of the instructors was found to be between 2.08±1.06 and 4.06±.72. Considering all the sub-dimensions of NUCEF, students think that nursing undergraduate curriculum is partially adequate, and instructors think that nursing undergraduate curriculum is adequate. To conclude, it is recommended that the current undergraduate curriculum be reviewed and regulated, continuous and systematic program evaluation and improvement studies be carried out, and program evaluation studies in nursing education be increased.

Preschool Teacher Education As An Example Of Open Social System

With the realization of the substantial long-term educational, social, and economic benefits produced by high-quality preschool education, preschool teachers’ education is becoming more focused on in recent years. Moreover, other than the studies focus on providing evidence for the impact of the preschool education on the children outcomes, studies showing large benefits occur only when teachers are professionally prepared and adequately compensated are also increased. On the other hand, studies focusing on the program evaluation or educational systems mainly consider the issue from a more structured and multidimensional view. In this respect, as an example of open social systems, teacher education depends on many interrelated components that affect each other directly or indirectly. The aim of this study is to review and discuss some of these components of preschool teacher education Turkish context such as the student selection policy, faculty development, recruitment policy and competencies of the graduates. Current issues of preschool teacher education are reviewed within the light of the research and related documents in order to understand the context within a macro perspective.Keywords: Preschool teacher education, educational policy, open social system, Turkish context

Adapting an Integrated Program Evaluation for Promoting Competency‐Based Medical Education

Educational program evaluation can improve the quality of the curriculum, instructional methods, and resources and provide useful data for making educational decisions and policies. Developing and implementing a program evaluation system is essential in competency-based medical education. The purpose of this study was to explore and establish an educational program evaluation system adapting an integrated program evaluation model to promote competency-based medical education. First, an Educational Evaluation Committee was organized, consisting of faculty, staff members, and students. The committee established an integrated program evaluation model, combining Stufflebeam’s Context, Input, Process, and Product (CIPP) model of a process-oriented approach and Kirkpatrick’s four-level model of an outcome-oriented approach. Kirkpatrick’s model was applied to the product evaluation of the CIPP model. The committee then developed evaluation criteria, indicators, and data collection methods according to the components of the CIPP model and the four levels (reaction, learning, behavior, and results) of Kirkpatrick’s model, and collected and analyzed data. Finally, the committee reported the results of evaluation to a Medical Education Quality Improvement Committee, and the results were used to improve the curriculum and student selection. To enhance the quality of education, identifying educational deficiencies and developing various elements of education in a balanced way through educational evaluation will be needed. Furthermore, it will be necessary to listen to opinions of various stakeholders, work with all members involved in education, and communicate with decision-makers in the process of educational evaluation.

Educational Program Evaluation System in a Medical School

A systematic educational program evaluation system for continuous quality improvement in undergraduate medical education is essential. Monitoring and evaluation (M&E) are two distinct but complementary processes referred to in an evaluation system that emphasizes formative purpose. Monitoring involves regular data collection for tracking process and results, while evaluation requires periodic judgment for improvement. We have recently completed implementing an educational evaluation using the M&E concept in a medical school. The evaluation system consists of two loops, one at the lesson/course level and the other at the phase/graduation level. We conducted evaluation activities in four stages: planning, monitoring, evaluation, and improvement. In the planning phase, we clarified the purpose of evaluation, formulated a plan to engage stakeholders, determined evaluation criteria and indicators, and developed an evaluation plan. Next, during the monitoring phase, we developed evaluation instruments and methods and then collected data. In the evaluation phase, we analyzed results and evaluated the criteria of the two loops. Finally, we reviewed the evaluation results with stakeholders to make improvements. We have recognized several problems including excessive burden, lack of expertise, insufficient consideration of stakeholders’ evaluation questions, and inefficient data collection. We need to share the value of evaluation and build a system gradually.

Evaluation of microwave-assisted ultraviolet digestion method for rice and wheat for subsequent spectrometric determination of As, Cd, Hg and Pb

The aim of this study was to optimize a sample preparation method for rice and wheat using microwave-assisted ultraviolet digestion (MW-UV) for subsequent determination of toxic elements. Cadmium and Pb were determined by sector-field inductively coupled plasma mass spectrometry (SF-ICP-MS), while As and Hg were determined by chemical vapor generation coupled to atomic absorption spectrometry (CVG-AAS). A systematic evaluation of the microwave heating program, nitric acid concentrationand sample mass was performed to optimize the MW-UV digestion method for rice and wheat samples. The relationship between nitric acid concentration and sample mass was monitored by determination of the residual carbon content (RCC) and residual acidity (RA) in order to obtain a high efficiency of digestion. The MW-UV method was successful at digesting up to 1100 mg of rice and wheat using 4 mol L−1 HNO3 with RCC and RA lower than 1.5 %, and 10 %, respectively. Recovery results ranging from 88 % to 117 % and agreement with certified reference values (t-test, 95 % confidence level) were obtained after digestion using the MW-UV method for spiked samples and certified reference materials (peach leaves-NIST 1547 and tomato leaves-NIST 1573), respectively. The optimized method was suitable for analysis of toxic elements in rice and wheat in compliance with the maximum levels reported in official directives.

Knowledge, attitudes and practices of COVID-19 among income-poor households in the Philippines: A cross-sectional study.

BackgroundThe presence of COVID-19 in low- and middle-income countries (LMICs) is raising important concerns about effective pandemic response and preparedness in the context of fragile health systems and the pervasiveness of misinformation. The objective of this study was to gain an understanding of how COVID-19 was perceived by households experiencing extreme poverty in the Philippines.MethodsThis study was conducted in partnership with International Care Ministries (ICM), a Philippine-based non-governmental organization (NGO) that runs a poverty-alleviation program called Transform targeted towards extreme low-income households. We integrated knowledge, attitudes, and practices (KAP) questions into ICM’s cross-sectional program monitoring and evaluation systems from February 20 through March 13, 2020. Frequencies and proportions were calculated to describe the respondents’ responses, and the Kruskal-Wallis test and binomial logistic regression were undertaken to determine the socio-demographic characteristics associated with COVID-19 KAPs.ResultsIn total, 2224 respondents from 166 communities in rural, urban and coastal settings were surveyed. Although the survey was administered during the earlier stages of the pandemic, 94.0% of respondents had already heard of COVID-19. Traditional media sources such as television (85.5%) and radio (56.1%) were reported as the main sources of information about the virus. Coughing and sneezing were identified as a transmission route by 89.5% of respondents, while indirect hand contact was the least commonly identified transmission route, recognized by 72.6% of respondents. Handwashing was identified by 82.2% of respondents as a preventive measure against the virus, but social distancing and avoiding crowds were only identified by 32.4% and 40.6%, respectively. Handwashing was the most common preventive practice in response to COVID-19, adopted by 89.9% of respondents. A greater number of preventive measures were taken by those with more knowledge of potential transmission routes.ConclusionsThere is a need for targeted health education as a response strategy to COVID-19 in low-income settings, and it is important that strategies are contextually relevant. Understanding KAPs among populations experiencing extreme poverty will be important as tailored guidance for public health response and communication strategies are developed for LMICs.

Ege Üniversitesi Tıp Fakültesi Öğrenci ve Öğretim Üyesi Değerlendirme Formları ile Yapılan Ölçümlere İlişkin Geçerlilik - Güvenilirlik

Introduction Program evaluation in medical education provides data that guide the program development process. It is expected that evaluation instruments are valid, reliable and low-cost, and useful for obtaining opinions of different groups such as student, teacher or graduate. Program evaluation has been a crucial component of program development in Ege University School of Medicine since 2001. Materials and Methods In this methodological design type study, reliability and validity of the results, obtained by updated students’ and teachers’ evaluation forms, were assessed. Cronbach alpha coefficient was calculated for assessing internal consistency reliability. Item content validity ratios were calculated in an expert panel. Exploratory factor analysis was implemented for determining construct validity. Interrater agreement and interrater reliability related to interpretation and use of the judgments of different observers were analyzed. Consistency was examined by a two-way mixed-effect model at 95% confidence interval, for calculating the intraclass correlation coefficient (ICC) related to interrater reliability. ResultsCronbach alpha coefficients that were over 0,7, except for Second and Third Year Student Block Evaluation Form, corroborated the internal consistency reliability for updated student and teacher evaluation forms. In construct analysis, student forms, except the Second and Third Year Block Evaluation Form were found to have a one-dimension construct, while teacher forms were determined to have a three-dimensions construct.Reliability of measurement of Factor II of Second and Third Year Block Student Forms and Factor III of Teachers’ Evaluation Forms was not corroborated as the interrater agreement and reliability coefficients were below acceptable levels. Results of this study revealed the evidences that support the validity and reliability of measurements done by the current evaluation forms used in the program evaluation system of Ege University School of Medicine, and also detected the points to be paid attention while interpreting and using the results of the measurements.

Provision of resource facilitation services for individuals with acquired brain injury across the United States: results of a 2018 resource facilitator provider survey

ABSTRACT Objective Resource Facilitation is a lesser-known service line within the field of brain injury rehabilitation and has been described as similar to case management, care coordination, and neuronavigation. The purpose of this project was to evaluate current Resource Facilitation programs and provide a comprehensive summary of program characteristics to work toward a common definition of Resource Facilitation services and inform future program development and evaluation frameworks. Materials and Methods An online survey was sent to all known Resource Facilitation programs in the US (N = 70). Data Collection Completed through Survey Monkey including information related to program history, structure, population served, and outcomes measurement. StatisticalAnalysis Descriptive statistical analyses were applied. Results Twenty-four of the 70 programs completed the survey, resulting in a 34% completion rate. This snapshot of the current definitions and activities demonstrated high variability across programs regarding structure, funding, eligibility, and data collection. Only 33% of programs reported having a program evaluation system. Conclusions This project provided a comprehensive summary of Resource Facilitation program characteristics and associated outcome metrics. The findings may be used to begin the formulation of a common definition of Resource Facilitation services, a program evaluation framework, and aid in establishing a common data set across programs.

American Association for the Surgery of Trauma Prevention Committee topical update: Impact of community violence exposure, intimate partner violence, hospital-based violence intervention, building community coalitions and injury prevention program evaluation.

An effective injury prevention program is an important component of a successful trauma system. Maintaining support for a hospital-based injury prevention program is challenging, given competing institutional and trauma program priorities and limited resources. In light of those pressures, the American College of Surgeons Committee on Trauma mandates that trauma centers demonstrate financial support for an injury prevention program as part of the verification process, recognizing that hospital administrators might see such support as discretionary and ripe as a target for expense reduction efforts. This Topical Update from the American Association for the Surgery of Trauma Injury Prevention Committee focuses on strategies to be more effective with the limited resources that are allocated to hospital-based injury prevention programs. First, this review tackles two of the many social determinates of violence, including activities aimed at mitigating the impact of both community violence exposure and intimate partner/domestic violence. Developing or participating in coalitions for injury prevention, both in general with any injury prevention initiative, and specifically while developing a hospital-based violence intervention program, efficiently extends the hospital's efforts by gaining access to expertise, resources, and influence over the target population that the hospital might otherwise have difficulty impacting. Finally, the importance of systematic program evaluation is explored. In an era of dwindling resources for injury prevention, both at the national level and the institutional level, it is important to measure the effectiveness of injury prevention efforts on the target population, and when necessary, make changes to programs to both improve their effectiveness and to assist organizations in making wise choices in the use of their limited resources.

HIV Partner Service Delivery Among Blacks or African Americans - United States, 2016.

Identifying persons with human immunodeficiency virus (HIV) infection who are unaware of their infection status, linking them to HIV care, and reducing racial/ethnic disparities are important national HIV prevention goals (1). Blacks/African Americans (blacks)* are disproportionately affected by HIV infection in the United States. Although blacks represent 13% of the U.S. population (2), in 2017, 44% of diagnoses of HIV infection were in blacks, and the rate of new diagnoses in blacks (41.1 per 100,000 persons) was approximately eight times that of non-Hispanic whites (5.1) (3). HIV partner services are offered by health officials to persons with diagnosed HIV infection (index patients) and their sex- or needle-sharing partners, who are notified of their potential HIV exposure and offered HIV testing and related services (4). CDC analyzed 2016 data from the National HIV Prevention Program Monitoring and Evaluation system submitted by 59 health departments.† Among 49,266 index patients identified as potential candidates for partner services, 21,191 (43%) were black. The percentage of black index patients interviewed for partner services (76%) was higher than that for all index patients combined (73%). Among the 11,088 black partners named by index patients, 78% were notified of their potential HIV exposure. Fewer than half (47%) of those notified were tested for HIV infection. Among those tested, one in six (17%) received a new HIV diagnosis. The prevalence of newly diagnosed HIV infection was particularly high among black partners who were gay, bisexual, and other men who have sex with men (MSM) (37%) and transgender persons (38%). Effective implementation of partner services is important to identify HIV infection, link patients to care or reengage them in care, and provide prevention services to reduce HIV transmission.

An on-site health home for homeless men in addiction recovery.

Patients with comorbid substance abuse, mental health disorders, and chronic health conditions face difficult challenges when attempting to access health care. A large gospel rescue mission developed an onsite wellness center, based on an integrated, embedded practice health home model, to promote health care access among homeless men in addiction recovery. The center model integrated primary care with behavioral health, specialty care referrals, and wellness services to address the multifaceted needs of the population. Standardized health care performance measures and resident and staff feedback support the effectiveness of the model in promoting improved health care access and treatment adherence. Future directions for program expansion include infrastructure development, expanded community outreach, and evaluation of effectiveness of the onsite health home through systematic program evaluation and research.

Impact of programme evaluation through self-assessment in higher education institutions in Pakistan

Quality assurance in higher education in Pakistan was formally initiated when Quality Assurance Agency (QAA) was established under Higher Education Commission of Pakistan. The current study is a descriptive study which was conducted to review the impact of programme evaluation on Pakistani universities. Data available with QAA, Pakistan, Self-Assessment Report available for the programmes and field notes were used as tool in this study. Programme evaluation reports were graded on a rubric in order to rank departments within a university. The study shows that quality assurance mechanism has got its firm roots at micro level, that is, at university level in Pakistan under the supervision of QAA of Pakistan. The study would be of interest for all educationists as it shows both the role of QAA, Pakistan and the role of quality enhancement cells whose combined efforts have resulted into a systematic programme evaluation in Pakistani universities.
 Keywords: Programme evaluation, self-assessment, learning outcomes, feedback.

A Atuação dos(as) Secretários(as) na Gestão dos Programas de Pós-Graduação Stricto Sensu do Sul do Brasil

O artigo discute sobre a atuação dos profissionais do secretariado no contexto da gestão dos Programas de Pós-graduação (PPG) em Educação na região Sul do Brasil. A pesquisa teve como objetivo investigar a atuação profissional dos secretários ou secretárias dos Programas de Pós-graduação em Educação da Região Sul do Brasil de modo a reunir informações que pudessem dar visibilidade à ação destes profissionais do Secretariado. Da mesma forma, busca subsidiar uma reflexão desses profissionais e dos próprios programas acerca de suas práticas laborais, visando ao aprimoramento dos processos e práticas de gestão no âmbito da Pós-graduação stricto sensu em educação a partir da visão da Secretaria. A metodologia consistiu na realização de um levantamento tipo survey exploratório-descritivo. A pesquisa se justificativa porque a gestão do capital humano dedicado à produção da pesquisa no contexto da pós-graduação não tem sido analisada sob o ponto de vista das competências dos/as Secretários/as. Como resultados, apresentamos à comunidade acadêmica e tecemos breves reflexões sobre uma primeira lista com as principais atividades realizadas pelos/as secretários/as em relação aos quesitos da Plataforma Sucupira que subsidia o sistema de avaliação da Pós-graduação em Educação no Brasil.

Construction of employee training program evaluation system of three exponential forecast based on sliding window

Construction of employee training program evaluation system of three exponential forecast based on sliding window

A critical review of the early childhood program evaluation system

A critical review of the early childhood program evaluation system

Construction of employee training program evaluation system of three exponential forecast based on sliding window

One construction method for employee training program evaluation system of three exponential forecast based on sliding window has been proposed to improve the effectiveness of construction process for employee training program evaluation system. Firstly, construct employee training program evaluation forecast model with gray model and set up regression forecast objective function; secondly, construct sliding window by setting segmentation points and make real-time segmentation for gray model data by combining three smoothness exponential algorithm, attain real-time statistical feature of data, construct the function relation between sequence error forecast and compression ratio, make use of error forecast sequence to make segmentation point judgment and improve robustness of test. Lastly, the effectiveness of the algorithm has been verified through simulation experiment.

Evaluating Teacher Preparation Programs With Teacher Evaluation Ratings: Implications for Program Accountability and Improvement

States are incorporating evaluation ratings into new, multioutcome teacher preparation program (TPP) evaluation systems, yet little is known about the relationships between TPPs and the evaluation ratings of program graduates. To address this gap, we use teachers’ ratings on the North Carolina Educator Evaluation System to determine whether TPPs are associated with the evaluation ratings of their initially prepared teachers. We find that (a) teachers from certain TPPs have significantly different evaluation ratings than graduates of other programs; (b) it is important to adjust for elements of school context when analyzing the evaluation ratings of program graduates; (c) evaluation ratings are a function of preparation quality, not just selection into programs; and (d) evaluation ratings provide evidence on the performance of TPPs that is distinct from value-added. Our results indicate that evaluation ratings can be a valuable component of TPP evaluation systems and provide guidance for incorporating ratings into such systems.

Evaluating complex interventions: A theory-driven realist-informed approach

There is a growing recognition that programs that seek to change people’s lives are intervening in complex systems, which puts a particular set of requirements on program monitoring and evaluation. Developing complexity-aware program monitoring and evaluation systems within existing organizations is difficult because they challenge traditional orthodoxy. Little has been written about the practical experience of doing so. This article describes the development of a complexity-aware evaluation approach in the CGIAR Research Program on Aquatic Agricultural Systems. We outline the design and methods used including trend lines, panel data, after action reviews, building and testing theories of change, outcome evidencing and realist synthesis. We identify and describe a set of design principles for developing complexity-aware program monitoring and evaluation. Finally, we discuss important lessons and recommendations for other programs facing similar challenges. These include developing evaluation designs that meet both learning and accountability requirements; making evaluation a part of a program’s overall approach to achieving impact; and, ensuring evaluation cumulatively builds useful theory as to how different types of program trigger change in different contexts.

Paying it forward: Four-year analysis of the Eastern Association for the Surgery of Trauma Mentoring Program.

Mentorship programs in surgery are used to overcome barriers to clinical and academic productivity, research success, and work-life balance. We sought to determine if the Eastern Association for the Surgery of Trauma (EAST) Mentoring Program has met its goals of fostering academic and personal growth in young acute care surgeons. We conducted a systematic program evaluation of EAST Mentoring Program's first 4 years. Demographic information was collected from EAST records, mentorship program applications, and mentee-mentor career development plans. We reviewed the career development plans for thematic commonalities and results of a structured, online questionnaire distributed since program inception. A mixed methods approach was used to better understand the program goals from both mentee and mentor perspectives, as well as attitudes and barriers regarding the perceived success of this career development program. During 2012 to 2015, 65 mentoring dyads were paired and 60 completed the program. Of 184 surveys distributed, 108 were returned (57% response rate). Respondents were evenly distributed between mentees and mentors (53 vs. 55, p = 0.768). In participant surveys, mentoring relationships were viewed to focus on research (45%), "sticky situations" (e.g., communication, work-life balance) (27%), education (18%), or administrative issues (10%). Mentees were more focused on research and education versus mentors (74% vs. 50%; p = 0.040). Mentees felt that goals were "always" or "usually" met versus mentors (89% vs. 77%; p = 0.096). Two barriers to successful mentorship included time and communication, with most pairs communicating by email. Most respondents (91%) planned to continue the relationship beyond the EAST Mentoring Program and recommended the experience to colleagues. Mentee satisfaction with the EAST Mentoring Program was high. Mentoring is a beneficial tool to promote success among EAST's young members, but differences exist between mentee and mentor perceptions. Revising communication expectations and time commitment to improve career development may help our young acute care surgeons.