Searching for the Difference: A Controlled Test of Just-in-Time Teaching for Large-Enrollment Introductory Geology Courses

Abstract

At Western Washington University (WWU), we tested the effectiveness of Just-in-Time Teaching (JiTT) methods in introductory geoscience courses. JiTT is an interactive-engagement teaching method developed for introductory physics courses (Novak et al., 1999). The key to the JiTT method is a short time frame feedback loop in which student learning informs instruction. The JiTT method includes several distinct components. (1) Web-based WarmUp exercises check on students' comprehension of reading assignments and ability to connect new concepts to those previously developed. Students submit answers to these exercises via the web up to four hours before each lecture. (2) Interactive lectures based on WarmUp responses emphasize problem-solving skills. (3) Web-based puzzles wrap up topics with critical thinking problems. We conducted our test using two sections of Geology 101 at WWU: the experimental section applied JiTT methods and the control section was taught using the traditional lecture format. Our experimental and control sections were non-major general education courses, had the same learning objectives, and objectives were assessed using the same exams. Students completed a First Day Survey providing baseline data about their demographics, science courses taken previously in high school and college, their attitude about studying science and their knowledge about basic Earth science concepts. On the first exam, the JiTT students scored significantly higher than the students in the traditional lecture section by an average of 2 points out of 50. The JiTT students' combined average on three exams during the course was higher, but not statistically significant. When the data from the First Day Survey were taken into account using multi-variable regression analysis, all differences in performance on exams becomes insignificant. We interpret our results to indicate that scores on objective (emphasizing knowledge and comprehension) exams may depend strongly on students' incoming prior knowledge, more than on the specific methods used to teach the class. Student evaluations of the course taught using JiTT methods were very positive toward the JiTT methods and their experience with introductory geology. Although we hope that such affective differences foreshadow long-term enhancement in learning (that our study did not assess), we caution instructors not to rely only on exam scores or student evaluations to gauge teaching effectiveness. Conversely, we recognize that scores on objective exams may not capture development of higher order thinking that the JiTT method may foster.