Retrospective Pretest Research Articles

Developing a Public Health Course to Train Undergraduate Student Health Messengers to Address Vaccine Hesitancy in an American Indian Community.

The purpose of the Diné Teachings and Public Health Students Informing Peers and Relatives about Vaccine Education (RAVE) project was to develop strategies for health communication that addressed COVID-19 vaccine safety for residents of the Navajo Nation. The RAVE project developed a 16-week course using the Diné Educational Philosophy as a framework to train Diné College (DC) public health undergraduate students (n = 16) as health messengers to share COVID-19 vaccine safety information with unvaccinated peers and relatives. An online community survey (n = 50) was used to assess DC community vaccination perceptions to guide course development. The two primary reasons survey participants got vaccinated were to protect the health of others [82% (n = 41)] and to protect their own health [76% (n = 38)]. A pretest/post-test and a retrospective pretest (n = 13) were implemented to determine course effectiveness. A finding approaching significance was related to student confidence in being health messengers (9.1% increase). RAVE offers the first example in the published literature of successfully training American Indian undergraduate students in the context of a public health course to contribute to the response workforce during a public health crisis.

K-12 Special Education Accessibility and Instructional Quality Related to the COVID-19 Shift to Online Instruction

ABSTRACT The shift to online teaching dictated by the COVID-19 pandemic was disruptive to all levels of education. Research has shown that access to online instruction was particularly challenging for K-12 students with disabilities and that their teachers were not well-prepared to provide instruction online. The objectives of this study were to evaluate the relationships between special education teacher characteristics and the switch to online instruction related to the COVID-19 pandemic as predictors of quality of instruction, and also to explore how the switch to online instruction was related to teacher and student access to the content and other relevant resources. Using a mixed methods online survey that incorporated a retrospective pretest approach for quantitative data collection, data from 293 special education teachers were used to examine these relationships. Regarding quality of instruction, paired samples t-tests yielded a large and significant drop in Engagement, a smaller drop in Action and Expression, and a small but significant increase in Representation. Teacher experience was significantly and positively correlated with student access to technology. Teachers described poor institutional support for the transition to online instruction and also reported problems with accessibility for their students. Only 22 of the 293 participants reported any prior experience with online teaching. Future research should include development of teacher preparation that includes training in improving Engagement as well as practice with online teaching skills and resources.

Implementation of a Mental Health Nursing Practicum in an Urban Homeless Shelter.

People experiencing homelessness suffer from deficient access to health care and disproportionately poor health outcomes. The American Association of Colleges of Nursing (AACN) maintains learning competencies for prelicensure nursing students. Shelters are rich environments for students to garner experiences with the inequities plaguing our health care system and to fulfill AACN competencies. We established a psychiatric and mental health nursing practicum at a homeless shelter. Following a retrospective pretest methodology, we evaluated student learning with the Health Care Professional's Attitudes Toward the Homeless Inventory (HPATHI). Students, faculty, and shelter staff provided qualitative feedback postpracticum. Students' median HPATHI scores increased on 16 of 19 survey items. Qualitative feedback was largely positive and reinforced HPATHI data. The practicum provided rich learning experiences for students. Educators establishing shelter-based practica should prioritize continuity, develop referral pathways for residents with illness, maintain a flexible mindset, administer prospective student surveys, and solicit feedback from shelter residents. [J Nurs Educ. 2024;63(X):XXX-XXX.].

Impacts of a Poverty Simulation Experience on Student Empathy Levels Toward People Living in Poverty

Many people hold negative opinions about individuals living in poverty without direct experience with poverty. Educating college students on the realities of life in poverty is critical to prepare them to serve in human-service careers. One way to educate about poverty is through an experiential education tool called a “simulation.” Poverty simulations are powerful tools that give participants embodied role-play experiences to promote social empathy and combat negative or singular perceptions of poverty. We explore the impact of the Missouri Community Action Network poverty simulation (CAPS) on student empathy for people experiencing poverty. We used the Undergraduate Perspectives of Poverty Tracking Survey (UPPTS) implemented as a retrospective pretest. While the impacts of CAPS have been studied, our unique contribution is the focus on empathy and use of the UPPTS. We found that student empathy for people living in poverty increased after the simulation with the greatest change occurring around four UTTPS factors: welfare attitudes, do more, lack of resources, and equal opportunity. Gender and financial status were the two demographics that had statistically significant difference of impact between categories with women and those who were “very financially secure” experiencing a greater change in empathy between pre and post scores.

Designing a questionnaire with retrospective pre-post items: Format matters

Evaluators are frequently asked to evaluate educational interventions that are one-time events. The Retrospective Pretest (RPT) methodology is well suited for these circumstances and the evaluation literature is replete with discussions about the pros and cons of this approach. In RPT, program participants rate their attitudes or knowledge now, and also rate how they were before participation in the intervention. The difference between now and before ratings constitutes the measure of change. Little published literature exists about whether the layout of RPT items within the evaluation questionnaire yields different results. This study compared six different layouts using a sample of 1941 caregivers who participated in one of 96 training workshops. The layouts were compared on inattentiveness, unexpected decline in perceived knowledge, and the degree of before-now change. Findings show that design matters. The best performing layout is where items are placed in the center with before response options on the left of the page and the now response options on the right. Results point to the need for evaluators to pay attention to RPT layout, and for the field to establish criteria for assessing survey layout quality. The unexpectedly high rate of inattentiveness calls for evaluators to pay more attention identifying and addressing this in their survey data.

Students’ perceived benefits of chess: Differences across age and gender

The purpose of this study was to determine differences in students’ perceived benefits of chess by gender and age after being exposed to chess during instruction for an entire academic year as part of a Chess in Schools initiative in a southeastern state of the United States. Data were collected during the 2017–201 8 academic year. The sample consisted of 1,286 students across elementary, middle, and high school levels. Data were collected using a retrospective pretest survey. Seven constructs of the students’ perceptions were generated (increased enjoyment of mathematics, improvement in academic self-efficacy, improvement in collaborative skills, improvement in organizational skills, enjoyment of chess, confidence in chess playing ability, and increased engagement in learning), and all had high reliability coefficients. Data were analyzed using a two-way multivariate analysis of variance. The results showed that elementary students consistently tended to have higher ratings of perceived benefits than middle and high school students across all constructs. The differences between middle and high school students were low and not statistically significant.

Training law students like athletes: experimenting with the constraints-led approach in law clinics

ABSTRACT We studied to what extent insights from the “constraints-led approach” (CLA) in physical education can be applied to skills teaching for law students in preparation for law clinics. In the CLA, athletes are challenged to independently find optimal movement solutions in a dynamic setting, rather than responding to verbal cues in a fixed environment. Similarly, we designed and implemented two CLA training exercises in preparation for law clinics aimed at implicit acquisition of problem analysis skills. Our analysis of retrospective pre-test (RPT) data finds that after the training exercises participants are better able to structure a clinical problem statement, and report being better equipped to find and analyse the “real” problem of a client in a clinic. We conclude that the CLA is an effective method for skill acquisition training in preparation for clinical legal education.

A comparative study of perspectives on telepractice service delivery among Egyptian and U.S. Speech and Language Pathologists

ABSTRACT Scope: Although the discipline of Speech-Language Pathology (SLP) was established in Egypt over 50 years ago, telepractice is a new concept in Egypt. However, following the COVID-19 pandemic providers adapted to an online therapy format. Materials & methods: The current study used a retrospective pretest–posttest design to identify differences in barriers facing SLPs practicing in the USA (N = 38) and Egypt (N = 41) using telepractice. The online questionnaire assessed the domains of technical barriers, the applicability of various types of therapy, caseload characteristics’ barriers, caregivers’ challenges, and the therapists’ input. Results: Therapists in both countries had concerns pre-telepractice regarding the availability of materials for online sessions, time spent for online session preparation, guaranteed confidentiality, and protection of client personal details in teletherapy. Most US therapists held a positive attitude towards the adaptability of various types of therapy for online sessions and the applicability of online sessions for new cases, toddlers & infants, preschoolers, adults & group sessions. On the other hand, Egyptian therapists expressed concerns regarding technical barriers such as lower frequency of sessions and internet connectivity problems while also experiencing more issues related to caseload characteristics. Conclusion: This study offers preliminary data on telehealth experiences in USA and Egypt that may direct future research into ways to increase service delivery and access.

Self-efficacy of Personal Finance Professionals in Later Life Financial Well-Being: Evidence From a Virtual Professional Development Seminar

Financial health falls under the purview of family and consumer sciences (FCS) educators and financial counselors. It is important for these professionals to understand the complexities of later life financial concerns in order to help prepare individuals at all stages of life for financial health in their senior years. We examined financial professionals' perceived mastery of later life finances by role and gender. We analyzed the effectiveness of a virtual professional development program in improving participants' perceived financial knowledge using retrospective pretest–posttest survey responses (n = 112). Results suggest that content focused, virtual professional development offerings may be an effective medium to enhance financial professional self-efficacy.

Athletic Training Preceptor Perceptions of Interprofessional Collaborative Practice in Clinical Learning Experiences

Background: Interprofessional education and collaborative practice are standards that have been added to the accreditation standards for athletic training education. Athletic training educators will need to provide interprofessional experiences to their students to remain compliant with the standards. The goal of the mixed methods study was to explore athletic training preceptors' perceptions of interprofessional education and collaborative practice in the clinical setting. Methods: A total of 34 athletic training preceptors completed the full online survey, and of those, 5 completed the semi structured interview. Participants completed an asynchronous online module and a retrospective pretest survey using the ISVS-21 about their beliefs, behaviors, and attitudes about interprofessional education and practice. Participants that volunteered were asked additional questions during a semi-structured interview. Results: The findings of this study found that the educational module was effective in increasing the beliefs, behaviors, and attitudes of athletic training preceptors with a short online module and that preceptors had positive attitudes/beliefs about IPE/IPCP, however, preceptors identified lack of education about IPE/IPCP and barriers to implementing IPE/IPCP with and without students that influenced their use of IPE/IPCP. Conclusion: Athletic training programs could educate and develop athletic training preceptors in interprofessional education and clinical practice to help develop opportunities for athletic training students in the clinical setting.

Validation of the Interprofessional Collaborative Competency Attainment Survey (ICCAS) retrospective pre-test measures

ABSTRACT Establishing the effectiveness of IPE experiences is essential, and current assessment measures may be subject to various types of bias. The Interprofessional Collaborative Competency Attainment Survey (ICCAS) is administered after the learning experience as retrospective pretest and posttest measures of perceived collaborative practice skills. Because the ICCAS does not have an actual pretest, there are potential risks of recall bias and social desirability bias. To address these concerns, Jackson (2017) proposed conducting a true ICCAS pretest before the learning experience and examining the differences between the true and retrospective pretest scores and the differences between these pretest and posttest scores. This study design was implemented for two in-person Interprofessional forums conducted in the Fall (N = 745) and following Spring (N = 599) semesters. Students from 11 (Fall) and 12 (Spring) health professions education programs were included. True ICCAS pretest scores did not significantly differ from the retrospective pretest scores in either sample, although the effect sizes for pretest to posttest gains were slightly smaller for the true pretest scores. These results support and provide confidence for using the standard ICCAS administration methodology by demonstrating the administration methodology is not adversely impacted by recall or social desirability biases.

Diné teachings and public health students informing peers and relatives about vaccine education: Providing Diné (Navajo)-centered COVID-19 education materials using student health messengers.

On 9 April 2021, the Centers for Disease Control and Prevention (CDC) reported that only 19. 9% of United States (US) adults were fully vaccinated against COVID-19. In that same week, the Navajo Nation (NN) reported that 37.4% of residents were fully vaccinated, making the NN a leader in the uptake of COVID-19 vaccines. Despite high vaccination rates, vaccine hesitancy exists within the NN. The Diné (Navajo) Teachings and Public Health Students Informing Peers and Relatives about Vaccine Education (RAVE) intervention was designed to utilize trusted health messengers as an effective means to address adults' vaccine concerns and hesitancy. The research team used COVID-19 vaccine materials developed in a previous collaboration with non-Navajo tribal communities and publicly available materials. Diné Traditional Knowledge Holders (TKHs) were interviewed to develop and incorporate Diné-specific information on individual and collective health behaviors into the RAVE materials. These drafted health education materials were presented to NN community health representatives (CHRs) and Diné public health students using a consensus panel approach. NN residents who participated in the intervention completed a 16-element retrospective pretest. The adaptation and tailoring process of materials yielded 4 health education materials. The students recruited 46 adults for health education sessions. These participants then completed the retrospective pretest. Changes in the 16 elements were in the desired direction, although only six were significant: four related to attitudes and two concerned with vaccination intention. Participants were more likely to consider vaccination and to try to get vaccinated after the education session. Trusted messengers and culturally centered materials have been identified as effective means of health behavior education with Native American audiences. RAVE applied these intervention elements by (1) training Diné College public health students to leverage their cultural knowledge and social relationships (cultural and social capital) to recruit vaccine-hesitant adults and provide education; (2) building on previous understanding of Native American communities' vaccine concerns; and (3) integrating Diné perspectives on individual and collective health into the adaptation of materials designed for general audiences; this knowledge was gained from interviews with TKHs.

To peer or not to peer: A controlled peer-editing intervention measuring writing self-efficacy in South Korean higher education

Peer editing has shown benefits for undergraduates writing in their native language. However, the results are not all positive when used in English as a Foreign Language (EFL) classes. There are concerns regarding the efficacy of L2 feedback and whether peer editing positively impacts learning outcomes. This article examines a control group and an experimental group of undergraduates enrolled in an online academic writing course at a medium-sized university in South Korea. The intervention involved three peer-editing assignments that were part of a larger essay project. The experimental group received training on how to edit and revise their peers’ compositions based on coded feedback. The researchers used a retrospective pretest posttest design to measure changes in writing self-efficacy. Quantitative results show that both groups significantly improved, but that neither group significantly improved more than the other. Qualitative findings from an open-ended post-semester survey suggest that peer editors reported they can learn through the process of editing by both the feedback they give and receive, and that social relatedness can also be enhanced, which is important in an online setting. However, participants also noted that peer feedback can be inaccurate and incorrect.

P089 A Retrospective Pretest-Posttest Design as an Alternative Evaluation Method for EFNEP

<h3>Background</h3> The Expanded Food and Nutrition Education Program (EFNEP) evaluates its behavioral outcomes using prospective pretest-posttests, which can be prone to response shift bias. A retrospective survey design can be a potential solution. <h3>Objective</h3> To examine program participants' understanding and preferences regarding retrospective versions of the EFNEP Adult Questionnaire (AQ). <h3>Study Design, Setting, Participants</h3> One-on-one, semi-structured interviews were conducted with 23 EFNEP adult participants/graduates in four US states/territories in 2020-2021. Participants were shown retrospective pretests and posttests in the "side-by-side" and "separate page" formats, which included one question from each of the AQ domains of diet quality, physical activity, food resource management, food safety and food security. AQ questions were modified by adding "Before EFNEP" for the pretest and "Now" for the posttest timeframes. <h3>Measurable Outcome/Analysis</h3> Demographic characteristics (e.g., age, gender, education), and AQ format preferences were collected. Audio transcripts were analyzed using template analysis and NVivo 1.5. <h3>Results</h3> Most of the participants were female (87%) and high school graduates (70%); their mean age was 35 years. For the "Before EFNEP" questions, participants mentioned "any time before the past two months" or provided the particular month before EFNEP lessons took place. Regarding the "Now" questions, emerging themes were relevant to the past weeks or past month. Some participants indicated that additional descriptions such as "in the week before EFNEP" were helpful in pinpointing the intended timeframe. Most (18/23) of the participants could correctly differentiate the time periods between the "Before" and "Now" questions. Most participants preferred the "side-by-side" (vs "separate page") format. <h3>Conclusions</h3> A retrospective AQ, especially in a side-by-side format, may be an appropriate alternative to the prospective surveys to evaluate behavioral outcomes of adult EFNEP participants. Next steps include piloting the retrospective survey to examine its criterion validity and feasibility with a larger sample. <h3>Funding</h3> Agricultural Experiment Station, NC3169 Multistate Project

Bad government performance and citizens’ perceptions: A quasi-experimental study of local fiscal crisis

The link between actual government performance and citizens’ performance perceptions has been controversial. Given the prevalence of negativity bias, however, the link between bad performance and citizens’ perceptions could appear to be strong. To explore this theoretically unconfirmed link, this study uses a quasi-experiment that contrasts a Japanese town in fiscal crisis, involving tax increases and service cuts, with a control village not in fiscal crisis. Using a difference-in-differences analysis with a careful retrospective pretest, it finds negative effects of the fiscal crisis on citizens’ process perception, while it shows no effects on citizens’ service satisfaction and trust in the mayor, council, and administrators. The study further finds positive associations between citizens’ performance perceptions and civic engagement. It discusses these findings to identify the boundary conditions in which a bad performance–negative perception link is likely to appear. Points for practitioners Psychology literature on negativity bias suggests that the causal links and mechanisms between bad performance and negative perceptions are stronger than those between good performance and positive perceptions. Not only citizens, but also politicians and administrators, hold negativity bias. Their blame-avoidance strategies could alleviate the growth of citizens’ negative perceptions with bad performance. Participatory governance might moderate the bad performance–negative perception link by placing citizens in a performance-improvement process and promoting their interaction with government officials.

Effects of the COVID-19 Pandemic on Parental Burnout and Parenting Practices: Analyses Using a Retrospective Pretest

Background Many of the conditions of the COVID-19 pandemic were consistent with factors shown to be predictive of parental stress and burnout. The purpose of the current study was to use a retrospective pretest method to gain an understanding of the effects of the COVID-19 pandemic on levels of parental burnout and on parenting practices. Method A brief survey was conducted using a retrospective pretest method to examine parental burnout (The Parental Burnout Assessment, Roskam et al, 2018) and parenting practices (The Alabama Parenting Questionnaire, Frick, 1991). The survey asked parent participants to answer questions about their experiences before and during the pandemic. Results Findings indicated that the pandemic had a significant impact on parents, increasing overall levels of parental burnout and impacting parenting practices by reducing use of positive parenting strategies and increasing use of inconsistent discipline and corporal punishment. These changes in parenting practices were even more pronounced for parents whose levels of parental burnout moved from “normal” levels before the pandemic to clinical levels during the pandemic. Conclusion The findings of the current study suggest that the COVID-19 pandemic has had a negative impact on levels of parental burnout and parenting practices. Although additional research is needed, the results suggest that there is a need for clinicians to understand the effects that the pandemic may have had on parents and families with an understanding that families may be at ongoing risk despite a relaxation of COVID-19 restrictions.

Using retrospective pretests to obtain subjective patient data: A case example.

In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

University students experience the COVID-19 induced shift to remote instruction

The COVID-19 pandemic required an abrupt shift from face-to-face to online instruction for many students in higher education in the United States. Prior research has raised some concerns about both equitable access to online courses, and the quality of instruction in online courses compared to face-to-face courses. This survey study included a retrospective pretest approach to comparing students experiences before and after the transition to online instruction. The sample of 1731 students ranged across all available topics of study and all class standings from first-year students to doctoral students at a R1: Doctoral Universities—Very High Research Activity university according to the Carnegie classifications. Quality of instruction was addressed through the three principles of Universal Design for Learning. Students reported that most areas of quality of instruction were poorer after the transition, with having Engagement dropping by the largest effect size. However, Representation showed a small effect of improvement following the transition. Students who preferred online instruction reported less loss of instructional quality. Similarly, students eligible for disability services also reported less loss of instructional quality. Doctoral students reported significantly poorer access on multiple measures compared to all four years of undergraduate students’ standings. Results are discussed in terms of patterns, exceptions, effect sizes, and recommendations for future research.

Vaccination induced complacency in adherence to COVID-19 precautionary measures among oral health care professionals in India and the United States: a retrospective pretest-posttest design

ABSTRACT In the context of the COVID-19 pandemic, vaccination-induced behavioral complacency in adherence to COVID-19 appropriate behavior emerged as a significant concern. This study was conducted among a convenience sample of 540 oral health care professionals in India and the United States. This was a retrospective pretest-post-test design, a choice to eliminate response-shift bias, where the participants responded online on their adherence or otherwise to COVID-19 precautionary measures before and after vaccination. The difference between post-test and retrospective pretest scores was used in assessing the magnitude of complacency demonstrated by the individual as a function of getting vaccinated, and the process was validated using exploratory factor analysis (EFA) with principal axis factoring and confirmatory factor analysis (CFA) on two randomly split subsets of the overall sample. It was observed that there had been a decline in the adherence to all the considered COVID-19 precautionary measures from the time before vaccination to the time of achievement of the fully vaccinated status. EFA performed on the randomly split sub sample of 240 subjects returned a two factor solution with five items in factor 1 and seven items in factor 2. Items in both the factors demonstrated adequate internal consistency in reliability analysis (Cronbach’s alpha 0.84 and 0.82, respectively). The two factor solution obtained in EFA demonstrated good model fit in CFA [RMSEA (90%CI) – 0.077 (0.063–0.092); TLI – 0.872; CFI – 0.897; SRMR – 0.056]. These results highlight the vaccination-induced complacency in observing COVID-19 appropriate behavior among oral health professionals in India and the United States.

Retrospective Pretests: Recent Use in Visitor Studies Research and Ways to Make Them More Informative

Many visitor studies researchers use Retrospective Pretest (RPT) ­methods to document outcomes. Research literature compares the validity of RPT with traditional pretest-posttests. This article reviews visitor study and informal learning literature about how RPT has been used to evaluate programs in museums, aquariums, parks/recreation, zoos, and tourism. It furthers discussions about response shift bias, arguing that response shift can be either intentional or inadvertent. The type of response shift is important in determining when RPT should be used. We argue that RPT is the best choice when the program intends to shift participants’ understanding of constructs being measured, and when comparing intervention with non-intervention groups. The literature review found that RPT results usually focus on statistical significance testing. RPT data offer much more learning when the results are also examined from other perspectives. This article describes additional analyses of RPT data that can assess the applicability of programs to intended audiences.