Pretest and posttest reliability to measure the impact of online team-based learning workshops during the Covid-19 pandemic as a continuing faculty development activity among health professional educators

Abstract

Medical education has been facing unprecedented challenges due to the COVID-19 pandemic leaving aftermath, which needs continuing reparative work. Imperative to exploring solutions faced by Covid pandemic challenges, various immediate steps were taken by medical schools globally including the faculty development activities transformed into online strategies. Workshops are the most common methods of the FDA to enhance knowledge and skills in specific areas of teaching and learning and assessment. A retrospective analysis of three of the TBL online FDA workshops is presented to determine its immediate impact on teaching and learning as well as an assessment model using pretest and posttest designs of a 3-point Likert scale of participants attending a half-day online workshop during the Covid19 pandemic for faculty development in a teaching hospital in Malaysia. Data was collected and analysed using online links created in Microsoft Google form at the beginning and towards the end of each workshop on a 3-point Likert scale of agreed, disagreed and not sure. A 15 items questionnaire was administered with additional information of participants’ trends in studying the reading materials sent out a week prior to the actual workshop. An interventional strategy between the two tests were four plenaries and a break-out session with hands-on group work followed by presentations. A collated quantitative data was gathered from 82 participants attending the online TBL workshops. Data were analysed using the statistical test of paired t-test, one-way ANOVA and ANCOVA with effect size in SPSS version 24. The percentage correct pretest and posttest score difference was significant at p = .013. The mean difference between pretest and posttest scores was significant at t = 31.345, p < 0.01. A significant difference in mean scores between groups was found, F (2,79) = 4.923, P = .010 using one-way ANOVA. Post-hoc analysis revealed significantly high posttest mean scores compared to partially read and not read groups at p = .033 and .014, respectively. However, the difference between partially read and not read was insignificant. A difference in mean posttest scores among the three groups using one-way ANCOVA on removing the effect of pretest was found significant at F (2, 80) = 6.211, p = 0.003. The effect size using Cohen’s d and Partial Eta Squared have a large practical effect of 2.072 and a moderate effect of 11.1% and 13.70% shown in paired t-test, ANOVA and ANCOVA respectively. A significant difference in the mean pretest and posttest scores was found between the groups. A post-test score, controlling on pretest score, was also significant, with a large Cohen’s d and moderate practical effect size suggesting an effectively delivered TBL online workshop. Keywords: Faculty development, online workshops, pretest, posttest design, quantitative evaluation, team-based learning