Retrospective Pre-test Post-test Research Articles

Impact of Seating and Mobility Services for Individuals With Disabilities in El Salvador.

Outcomes research on the impact of seating and mobility services delivered using a short-term medical mission (STMM) model is limited. To evaluate the impact of seating and mobility services on the occupational performance of individuals with disabilities in El Salvador. One-group retrospective pretest-posttest. Clinical (El Salvador). Individuals with disabilities in El Salvador. Survey and the standardized Wheelchair Outcome Measure (WhOM) for those who received a wheelchair as their mobility device. Participants rated satisfaction with performance of preferred in-home and out-of-home occupations on the WhOM before and after receiving seating and mobility services. For most survey questions assessing the impact on activities of daily living, more than 86% of the respondents selected agree or strongly agree. Nearly half of the respondents reported that questions regarding work and education were not applicable. Participants' WhOM scores (n = 86) demonstrated a statistically significant improvement in performance satisfaction for both in-home (p < .001; d > 1) and out-of-home (p < .001; d > 1) occupations after they received services, with a huge effect size (d > 2). The findings suggest that seating and mobility services provided by rehabilitation professionals in El Salvador improved occupational performance for people with disabilities. Compared with STMMs that solely provide equipment, the findings emphasize the importance of professional service provision with education and training as best practice. What This Article Adds: Seating and mobility services delivered through a STMM model may improve occupational performance for individuals with disabilities. However, STMMs should be carefully planned in collaboration with in-country partners, provide customized seating systems, and include education and training from licensed rehabilitation professionals.

To peer or not to peer: A controlled peer-editing intervention measuring writing self-efficacy in South Korean higher education

Peer editing has shown benefits for undergraduates writing in their native language. However, the results are not all positive when used in English as a Foreign Language (EFL) classes. There are concerns regarding the efficacy of L2 feedback and whether peer editing positively impacts learning outcomes. This article examines a control group and an experimental group of undergraduates enrolled in an online academic writing course at a medium-sized university in South Korea. The intervention involved three peer-editing assignments that were part of a larger essay project. The experimental group received training on how to edit and revise their peers’ compositions based on coded feedback. The researchers used a retrospective pretest posttest design to measure changes in writing self-efficacy. Quantitative results show that both groups significantly improved, but that neither group significantly improved more than the other. Qualitative findings from an open-ended post-semester survey suggest that peer editors reported they can learn through the process of editing by both the feedback they give and receive, and that social relatedness can also be enhanced, which is important in an online setting. However, participants also noted that peer feedback can be inaccurate and incorrect.

Impacts of COVID-19 Pandemic on Hospitality and Tourism Students

Tourism education is part of the complex tourism system as it provides professionally trained and qualified employees for the industry. Given the severe impacts of COVID-19 on tourism and the interconnected relationship between tourism education and the industry, the pandemic might have altered hospitality and tourism (HTM) students' perceptions toward the sector and, subsequently, their career intention. Against this backdrop, this study explores the pandemic's impacts on HTM students by analyzing the degree of change in their perceptions toward tourism and the effect of that potential perceptual change on their career intention before and after COVID-19. The study used a quantitative research design's retrospective pretest-posttest (RPP) model. Cluster sampling was used to draw the study sample, which comprised HTM students enrolled at several major HTM educational institutions in Malaysia. Descriptives were used to analyse the respondents' profile and program enrollment data, a t-test was used to examine the respondents' pre- and post-COVID-19 tourism perceptions, and correlation was used to explore the relationship between perception and career intentions. Key findings suggest that after COVID-19, HTM students have become less optimistic about the tourism industry and have lowered tourism-career intention after graduation. Additionally, such skills and knowledge as professional attitude and disciplinary skills might have become more critical and relevant. These findings underscore the importance of restoring HTM students' confidence and trust in tourism education and industry.

P089 A Retrospective Pretest-Posttest Design as an Alternative Evaluation Method for EFNEP

<h3>Background</h3> The Expanded Food and Nutrition Education Program (EFNEP) evaluates its behavioral outcomes using prospective pretest-posttests, which can be prone to response shift bias. A retrospective survey design can be a potential solution. <h3>Objective</h3> To examine program participants' understanding and preferences regarding retrospective versions of the EFNEP Adult Questionnaire (AQ). <h3>Study Design, Setting, Participants</h3> One-on-one, semi-structured interviews were conducted with 23 EFNEP adult participants/graduates in four US states/territories in 2020-2021. Participants were shown retrospective pretests and posttests in the "side-by-side" and "separate page" formats, which included one question from each of the AQ domains of diet quality, physical activity, food resource management, food safety and food security. AQ questions were modified by adding "Before EFNEP" for the pretest and "Now" for the posttest timeframes. <h3>Measurable Outcome/Analysis</h3> Demographic characteristics (e.g., age, gender, education), and AQ format preferences were collected. Audio transcripts were analyzed using template analysis and NVivo 1.5. <h3>Results</h3> Most of the participants were female (87%) and high school graduates (70%); their mean age was 35 years. For the "Before EFNEP" questions, participants mentioned "any time before the past two months" or provided the particular month before EFNEP lessons took place. Regarding the "Now" questions, emerging themes were relevant to the past weeks or past month. Some participants indicated that additional descriptions such as "in the week before EFNEP" were helpful in pinpointing the intended timeframe. Most (18/23) of the participants could correctly differentiate the time periods between the "Before" and "Now" questions. Most participants preferred the "side-by-side" (vs "separate page") format. <h3>Conclusions</h3> A retrospective AQ, especially in a side-by-side format, may be an appropriate alternative to the prospective surveys to evaluate behavioral outcomes of adult EFNEP participants. Next steps include piloting the retrospective survey to examine its criterion validity and feasibility with a larger sample. <h3>Funding</h3> Agricultural Experiment Station, NC3169 Multistate Project

Teacher agency and professional development: A study on CambridgeEnglishteacher program in theArabianGulf

Teacher agency has a powerful influence on teachers’ classroom practices, student learning outcomes, and other professional undertakings. Although teacher agency shaping is a core subject of professional development (PD), most in-service PD initiatives focus on developing teaching knowledge and skills while enhancing teacher agency. The current study investigates the influence of Cambridge English Teacher (CET) professional development on EFL teachers’ agency development in an English Language Institute at a Saudi Arabian public university. Utilizing retrospective pretest post-test design, a quantitative questionnaire provides data to understand EFL teachers’ agency development as they participate in the CET development program. Based on the theoretical framework of the current study, EFL teacher agency change is characterized by three attributes: teachers’ say in curriculum and syllabus designing, teachers’ freedom to implement teaching methods and techniques, and teachers’ autonomy in using supplementary materials. In analysing the PRE and POST data, descriptive statistics and Wilcoxon signed Rank test were employed. The findings show that all three teacher agency attributes are significantly influenced as a result of the CET development program. However, regarding the extent to which teacher agency attribute has enhanced, the results exhibit different degrees of change. The findings suggest that institutionally imposed professional development is useful for the development of teachers’ agency, provided PD initiatives are cognizant of teachers’ needs and institutional requirements. The findings of the current study have implications for EFL teachers, teacher trainers or professional development specialists, and language institutes’ administration.

Including Diverse Stakeholder Voices in Youth Character Program Evaluation

Although experts agree that diverse stakeholder inclusion enhances quality and equity in evaluation designs and implementation, diverse voices are often omitted. Particularly antithetical to principles of youth character development, evaluations for these programs should strive to include voices from various social, economic, community, and demographic perspectives. One innovative national evaluation capacity building initiative, the Partnerships for Advancing Character Program Evaluation (PACE) project, paired practitioners from youth programs in community-based organizations with evaluation professionals to enhance stakeholders’ roles in evaluation. PACE promoted stakeholder identification and inclusion through group exercises, partnership work, and coaching sessions. Using a mixed methods design with interviews, retrospective pretest–posttest surveys, and observational data, triangulated data addressed diverse stakeholders in the evaluation process, diverse perspectives on program performance, and connecting diverse input to evaluation design. Postprogram findings indicate that participants included more varied and diverse stakeholder perspectives in all the three areas. Implications for programs and evaluations are discussed.

Student outcomes of an international learning collaborative to develop patient safety and quality competencies in nursing

Background Patient harm is a global crisis fueling negative outcomes for patients around the world. Working together in an international learning collaborative fostered learning with, from and about each other to develop evidence-based strategies for developing quality and safety competencies in nursing. Aims To report student outcomes from an international learning collaborative focused on patient safety using the Quality and Safety Education for Nurses competency framework. Methods A global consortium of nursing faculty created an international learning collaborative and designed educational strategies for an online pre-workshop and a 10-day in-person experience for 21 undergraduate and graduate nursing students from six countries. A retrospective pre-test post-test survey measured participants’ confidence levels of patient safety competence using the health professional education in patient safety survey and content analysis of daily reflective writings. Results Statistical analysis revealed student confidence levels improved across all eight areas of safe practice comparing-pre and post-education (significance, alpha of P &lt; 0.05). Two overarching themes, reactions to shared learning experiences and shared areas of learning and development, reflected Quality and Safety Education for Nurses competencies and a new cultural understanding. Conclusions The international learning collaborative demonstrated that cross-border learning opportunities can foster global development of quality and safety outcome goals.

Capitalising on opportunities: Malnutrition coding in hospital before and after the introduction of electronic health records with an embedded malnutrition screening tool.

During conversion from paper to electronic documentation at a tertiary hospital, the Malnutrition Screening Tool was embedded into the electronic health record (e-HR) with nursing staff's completion continued as part of admission procedures with dietetic referrals automated. Currently, the impact of e-HR implementation on malnutrition identification is unknown. Consequently, this retrospective pre-test post-test study compared one year of malnutrition coding in a tertiary teaching hospital two years before and after e-HR implementation automating malnutrition screening referrals to dietitians with subsequent malnutrition assessment completion. Eligibility included adults (≥18yrs) admitted overnight or longer during the 2013/2014 and 2017/2018 financial years. Requested hospital data included demographics, admission data and coding for malnutrition and dietitian intervention. Eligible admissions prior to e-HR implementation were classified as pre-e-HR group, with admissions after classified as post-e-HR. Descriptive, Fisher's exact, Mann-Whitney U and independent samples t-tests were used to compare groups. Patient admissions pre-e-HR (n=37,143) and post-e-HR (n=36,625) implementation were clinically similar in age, gender and length of stay (57 ± 19 years, 60% male, 3 (1-918) days). However, the numbers of admissions coded annually with malnutrition increased by 47% from 1436 to 2116 following e-HR implementation (p<0.001). The proportion of eligible patients who were malnourished on admission and not seen by a dietitian during admission decreased one third from 6.5% to 4.8% (p=0.042). Malnutrition coding increased by 47% after an e-HR implementation with an embedded malnutrition screening tool that automated referrals to dietitians impacting the identification of care to optimize nutritional status.

LO81: The use of in situ simulation to improve emergency department staff comfort with the management of high acuity, low occurrence cases

Introduction: In the emergency department (ED), high-acuity presentations encountered at low frequencies are associated with reduced staff comfort. Previous studies have shown that simulation can improve provider confidence with practical skills and management of presentations in various fields of medicine. The present study examined the effect of in situ simulation on interprofessional provider comfort with the identification and management of high-acuity low-frequency events in the ED. It further assessed the feasibility of implementing weekly simulation as an interprofessional education initiative in a high-volume ED. Methods: This was a retrospective pre-test post-test quasi-experimental design. Weekly in situ simulation events were facilitated by an interdisciplinary team in a high-volume ED in Hamilton, Ontario that sees an average of 185 patients per day. To date, 34 simulation events were held between January 18, 2019 and November 22, 2019, and included neonatal, paediatric and obstetric emergencies, and adult codes. There was an average of 20 patients presenting to the ED during these events. Events included a debrief, and typically lasted 60 minutes in total. Participants included individuals from various disciplines working on shift at the time of the event. Questionnaires were administered via email following the event, in which participants were asked to rank their comfort with emergency codes before and after the simulation using two 5-point Likert scales. The data from 39 questionnaires was analyzed. T-tests were used to analyze differences in self-reported comfort scores. Results: Questionnaire responders included nurses (41%), respiratory therapists (26%), resident physicians (10%), paramedics (3%), attending physicians (3%), students of various disciplines (10%) and other (7%). 38% of participants reported increases in comfort following simulation when compared to prior. Using the 5-point scale, the average reported score for comfort pre-simulation was 3.59 (95% CI 3.30–3.88), and the average post-simulation score was 3.97 (95% CI 3.76–4.19, p = 0.03). Conclusion: Our results demonstrate that weekly interprofessional in situ simulation is feasible in a high-volume ED, and significantly improves self-reported provider comfort with the identification and management of high-acuity, low-frequency events. This warrants the implementation of this simulation design to improve staff confidence and has implications for its potential role in improving team dynamics and patient safety.

Reflective learning for behavioral guidance in pediatric dentistry.

Basic behavioral guidance (BBG) skills help in delivery of quality health care in pediatric dentistry. The complex nature of these skills, warrants analyzing actions performed. An appropriate scientific way to do this is by "reflection." Hence, the present study was designed to introduce "reflection of action" as means for learning BBG skills by undergraduates in pediatric dentistry. Participants (dental undergraduate) performed oral prophylaxis (two visits) in pediatric patients (age 3-7 years) with application of BBG skills, under video recording. They were instructed regarding "reflection on action." The learners then reviewed own videos and wrote reflections. Reflections were assessed on Boud's 4R framework and feedback was given by the faculty. Reflective writing was repeated for a second visit. Knowledge of the learners in using reflections for learning was assessed by retrospective pretest posttest questionnaire. Video recordings were scored for BBG skills. Acceptability of the intervention was addressed by satisfaction questionnaire. There was a significant improvement in the knowledge of participants in using reflections for learning these skills. All participants were able to "revisit" (R1 level under Boud's 4R framework) patient encounter in their written reflections. Sixteen participants exhibited shift toward higher levels in the next visit. Video scores of learners also improved significantly over both visits. Students were satisfied with the content, delivery, and relevance of the new educational intervention. The strong need of improving BBG skills in pediatric dentistry was met by "reflection on action." There was improvement in the knowledge of students in using reflections for learning and application of behavior guidance skills and was well accepted.

The retrospective pretest–posttest design redux: On its validity as an alternative to traditional pretest–posttest measurement

We revisit the merits of the retrospective pretest–posttest (RPP) design for repeated-measures research. The underutilized RPP method asks respondents to rate survey items twice during the same posttest measurement occasion from two specific frames of reference: “now” and “then.” Individuals first report their current attitudes or beliefs following a given intervention, and next they are prompted to think back to a specific time prior to the given intervention and rate the item again retrospectively. The design addresses many of the validity concerns that plague the traditional pretest–posttest design. Particularly when measuring noncognitive constructs, the RPP design allows participants to gauge the degree of change that they experience with greater awareness and precision than a traditional approach. We review the undesirable features of traditional designs and highlight the benefits of the retrospective approach. We offer examples from two recent, original studies and conclude with the recommendation that the RPP design be employed more broadly. We also conclude with a discussion of important directions for future examination of this design.

Structure and Psychometric Properties of the Father Involvement Scale Used in a Retrospective Pretest–Posttest Design

Several programs that address responsible fatherhood exist; however, these programs need strategic evaluation and appropriate tools to measure success of the programs. The goal of this research was to understand the structure and psychometric properties of a father involvement measure used to assess the change in parenting and co-parenting behaviors of fathers in a child welfare program. Analyses used secondary data from a father parenting program, with a retrospective pretest–posttest design. Participants were fathers (N = 361) who attended a parenting program in three large counties in the North Texas region. Missing data diagnosis and exploratory and confirmatory factor analyses were performed on 13 items to investigate scale structure and psychometric properties. The following three factors with adequate to good internal consistency emerged: self-efficacy, parenting (emotional and physical support), and co-parenting skills. Fit indices showed good model fit for pretest items and a moderate fit for posttest items. The survey instrument demonstrates potential to be adapted to diverse populations, other languages and cultures to further expand program components. Based on the findings, future research should utilize more items to enhance the alpha reliability of the scales. Furthermore, revised scales can be validated across diverse populations based on socioeconomic, ethnicity, and cultural backgrounds.

Addressing osteoarthritis pain among older adult women through aquatic therapy: A new look at shallow-water movement

This study evaluated a shallow-water intervention to reduce pain among women age 61 to 81 with knee and/or hip osteoarthritis. The study used a randomized-replicated single-case two-condition crossover design to determine which condition was more effective at relieving pain. One condition was functional movements without the use of a stationary pole; the other condition, identical movements with the pole. Data were collected through repeated pain measures, pre-test-post-test measures, and retrospective pre-test post-test measures. Pain data analysis with Microsoft Excel® ExPRT 2.0 AB program generated pre- and post-crossover charts. Visual analysis findings were substantiated with sensitivity analyses of retrospective pre-test and post-test data, showing no significance between the two conditions (p ≤ 0.05). Results indicated use of a pole might not be more effective than the movement program itself. For the individual and recreation therapist, this finding might translate to cost effective recreation therapy interventions addressing clinical outcomes as a result from engagement with an aquatic RT movement program.

A Higher-Calorie Refeeding Protocol Does Not Increase Adverse Outcomes in Adult Patients with Eating Disorders

BackgroundPatients with eating disorders (EDs) are often considered a high-risk population to refeed. Current research advises using “start low, go slow” refeeding methods (∼1,000 kcal/day, advancing ∼500 kcal/day every 3 to 4 days) in adult patients with severe EDs to prevent the development of refeeding syndrome (RFS), typically characterized by decreases in serum electrolyte levels and fluid shifts. ObjectiveTo compare the incidence of RFS and related outcomes using a low-calorie protocol (LC) (1,000 kcal) or a higher-calorie protocol (HC) (1,500 kcal) in medically compromised adult patients with EDs. DesignThis was a retrospective pre-test–post-test study. Participants/settingOne hundred and nineteen participants with EDs, medically admitted to a tertiary hospital in Brisbane, Australia, between December 2010 and January 2017, were included (LC: n=26, HC: n=93). The HC refeeding protocol was implemented in September 2013. Main outcome measuresDifferences in prevalence of electrolyte disturbances, hypoglycemia, edema, and RFS diagnoses were examined. Statistical analysis performedχ2 tests, Kruskal-Wallis H test, analysis of variance, and independent t tests were used to compare data between the two protocols. ResultsDescriptors were similar between groups (LC: 28±9 years, 96% female, 85% with anorexia nervosa, 31% admitted primarily because of clinical symptoms of exacerbated ED vs HC: 27±9 years, 97% female, 84% with anorexia nervosa, 44% admitted primarily because of clinical symptoms of exacerbated ED, P>0.05). Participants refed using the LC protocol had higher incidence rates of hypoglycemia (LC: 31% vs HC: 10%, P=0.012), with no statistical or clinical differences in electrolyte disturbances (LC: 65% vs HC: 45%, P=0.079), edema (LC: 8% vs HC: 6%, P=0.722) or diagnosed RFS (LC: 4% vs HC: 1%, P=0.391). ConclusionsA higher-calorie refeeding protocol appears to be safe, with no differences in rates of electrolyte disturbances or clinically diagnosed RFS and a lower incidence of hypoglycemia. Future research examining higher-calorie intakes, similar to those studied in adolescent patients, may be beneficial.

Innovations for Evaluation Research: Multiform Protocols, Visual Analog Scaling, and the Retrospective Pretest-Posttest Design.

In this article, we review three innovative methods: multiform protocols, visual analog scaling, and the retrospective pretest-posttest design that can be used in evaluation research. These three techniques have been proposed for decades, but unfortunately, they are still not utilized readily in evaluation research. Our goal is to familiarize researchers with these underutilized research techniques that could reduce personnel effort and costs for data collection while producing better inferences for a study. We begin by discussing their applications and special unique features. We then discuss each technique's strengths and limitations and offer practical tips on how to better implement these methods in evaluation research. We then showcase two recent empirical studies that implement these methods in real-world evaluation research applications.

A retrospective study of a multicomponent rehabilitation programme for community-dwelling persons with dementia and their caregivers

Introduction This study determined whether the multicomponent rehabilitation programme of a memory clinic had positive outcomes on ameliorating everyday functioning, quality of life, mood and behavioural disturbances of persons with dementia and reducing distress and burden of caregivers. Method A retrospective pre-test–post-test study without control group was conducted on the first cohort of persons with dementia ( n = 30) and their caregivers ( n = 30), who participated in a programme lasting for a maximum of 1 year with 25 1-hour counselling sessions. The assessment contained an evaluation of everyday functioning in basic, instrumental and advanced activities of daily living, cognition, mood, emotional and behavioural disturbances, quality of life and caregiver burden. Results Eight participants dropped out prematurely. For persons with dementia ( n = 22), participating in the programme did not improve everyday functioning and cognition but ameliorated quality of life significantly ( Z = –2.7, p = 0.006, 95% CI (.003–.005)) and stabilized mood, emotional and behavioural disturbances for 60% or more of them. For caregivers ( n = 22), the mild to moderate burden of care remained stable or got better for 63.6% of the caregivers. Conclusion This programme appears to be promising and valuable, and might reduce institutionalization rates. Future explorations are recommended to research how participants evolve and to investigate which participants responded in a positive way.

Positive Youth Development Life Skills Gained at the Iowa 4-H Youth Conference

Research suggests 4-H programs build Life Skills such as leadership, communication, citizenship and learning. However, 4-H programs vary from long-term on-going experiences to shorter, more intense opportunities. This paper discusses a program evaluation articulating the life skill development of participants in a 3-day residential State 4-H Conference on a Midwestern college campus. The Life Skills assessed were in the areas of leadership, citizenship, communication, and learning as part of overall Life Skill development. Participants were youth ages 14-18 years. A retrospective pretest-posttest was used to evaluate skill development and understanding. Analysis, including paired sample t-tests, indicated growth in each of the 12 common outcome measures evaluated. This study supports the importance of purposeful planning and youth engagement in the learning process to achieve desired life skill outcomes.

A Retrospective Pretest-Posttest Evaluation of a One-Time Personal Finance Training

Attempts to measure the impact of Extension programs have been implemented to varying effects for many years. Using data from several economics of personal finance workshops designed to teach high school teachers about a state's new personal finance performance standards, this article reports the results from of one type of evaluation, a retrospective pretest-posttest, that is useful for one-time, short duration workshops where the extra steps and additional costs of traditional pretest-posttest evaluation may not be warranted. A discussion of the benefits of retrospective pretest-posttest evaluations, including reduced response shift bias, concludes the article.

Influence of Objective Measures on Self-Reports in a Retrospective Pretest-Posttest Design

Two experiments were done to assess the influence on subjects’ self-ratings of (a) awareness of an objective measure and (b) taking a performance test. The design of both studies was a pretest-posttest design, including retrospective preratings. Subjects were psychology freshmen fulfilling a course requirement. In line with earlier research, data indicate that the retrospective pre-rating approach reflects actual improvement of performance more accurately than the conventional prepost comparisons. The response shift phenomenon— defined as a significant difference between mean pre-ratings and mean retrospective ratings—was demonstrated in both experiments. Awareness of an objective measure, however, tends to weaken the response shift phenomenon, whereas actually taking a performance test tends to strengthen it.

Internal Invalidity in Pretest-Posttest Self-Report Evaluations and a Re-evaluation of Retrospective Pretests

True experimental designs (Designs 4, 5, and 6 of Campbell &amp; Stanley, 1963) are thought to provide internally valid results. This paper describes five studies involving the evaluation of various treat ment interventions and identifies a source of inter nal invalidity when self-report measures are used in a Pretest-Posttest manner. An alternative approach (Retrospective Pretest-Posttest design) to measuring change is suggested, and data comparing its ac curacy with the traditional Pretest-Posttest design in measuring treatment effects is presented. Finally, the implications of these findings for evaluation re search using self-report instruments and the strengths and limitations of retrospective measures are discussed.