Flipping the Biostatistics Classroom, With a Twist

Abstract

For ecology faculty members not directly involved in pedagogical research, it can be daunting to decide among the many new tools and approaches available to improve teaching and learning. Problem-based learning (Edens 2000), blended learning (Garrison and Kanuka 2004), case-based teaching (Herreid et al. 2011), active learning (Bean 2011), MOOCs (Daniel 2012), and flipping the classroom (Bergmann and Sams 2008, Bergmann and Sams 2012, Fulton 2012, Tucker 2012, Bishop and Verleger 2013, Herreid and Schiller 2013) are just a few of the new methods being evaluated and implemented in educational theory and practice. Flipping the classroom is the process of moving traditional lecture content teaching to videos watched by students outside the class, while simultaneously moving activities such as homework and group projects into the classroom (Bergmann and Sams 2008, 2012). At our university, replacement of traditional introductory biology lectures with recorded “talking head” videos in the late 1970's and early 1980's was an abysmal failure, one that damaged the reputation of our department long after the practice was abandoned. How does “flipping the classroom” differ from this? In what context might it make sense to move content delivery outside the classroom, while moving homework and group exercises into the traditional lecture period? In Fall, 2013, we worked together (professor and graduate student teaching assistant) to implement a flipped classroom approach in a biostatistics course in the Biology Department at West Virginia University. The philosophy of this course that semester was, and always had been, that statistics is best learned by doing it – lots of it – using problems or full-blown ‘case studies,’ that go all the way from (1) identifying the question being asked, and the hypothesis being tested, (2) describing the experimental design used to address the hypothesis, (3) identifying the appropriate analysis of the data that would emerge from the experiment, (4) carrying out the analysis with a realistic data set using statistical analysis software (we use SAS JMP), (5) presenting the results in both text and graphical form, and (6) distinguishing between results and discussion type statements about the findings. A prerequisite for the course is an introductory statistics course, but in our experience this has equipped students with very little confidence in their ability to apply statistics in the real world. Thus, our goal for the course is to build the students' ‘statistical toolbox’ so that they are equipped to confidently design and perform biological research projects requiring statistics. The tools learned range from simple statistics such as correlation and linear regression, to multi-way factorial designs incorporating nesting. The full ‘bestiary’ of designs (Gotelli and Ellison 2013) with nominal or continuous independent and dependent variables was covered. In previous iterations of the course, the professor typically spent lecture periods covering content for 20 – 30 minutes of the 50-minute class period. For the remainder of the class, sample problems were given to the students to try, in preparation for doing weekly homework sets later outside of class. The class was held in a computer lab, and the professor could demonstrate how to use the statistics software to perform a given test while students followed along, then students could practice with the sample problem(s). Because about half the class period was spent working on problems, this class was already partially ‘flipped’ (=in that active, hands-on learning exercises were already part of the classroom experience). Typical class size was 15–25 students, each having a separate computer to work on during the class. Student understanding was assessed using the homework problem sets completed outside of class (60% of course grade), and also with unannounced quizzes given ca. every other week (30% of course grade). Due to the accretive nature of the course material, and the resulting detrimental effects of missing class, attendance was strongly incentivized (10% of course grade). We built in penalties for individual absences, but also rewards for overall class attendance (>90% threshold; lowest homework grade dropped). A graduate section of the course was taught simultaneously, with ca. 5 – 10 students participating. The text for both the undergraduate and graduate sections of the course was Sokal and Rohlf's Biometry (2012, 4th edition), and graduate students were also asked to purchase Gotelli and Ellison's Primer of Ecological Statistics (2013) or the equivalent if the student was not an ecologist. With the flipped classroom version of the course, textbooks were still required, but used more for reference than a vital source for lecture preparation and review. Two major course changes were implemented in 2013 to flip the classroom further. First, ca. 90% of the in-class lecture was moved to online videos that students were assigned to watch before class. Second, concurrently we developed low stakes online quizzes on the videos to incentivize watching them prior to class. There are many options available for developing both of these activities, and the most expeditious route will depend on the college or university support, as well as your facility with computers. In our case, videos were created using (1) pen input with an Intuos bamboo tablet attached to an iMac computer with Sketchbook Express software displaying a whiteboard encompassing the majority of the screen, (2) a small live video window showing the professor talking using iChat Preview mode, (3) often a picture or image showing something related to the lecture material, (4) QuickTime Screen Recording, (5) assembly and export of the video in iMovie, followed by, (6) uploading the compiled video to YouTube to a dedicated channel called ‘Biometry Online Lessons.’ Online quizzes were created using a facility in eCampus, which is our university's standard course software. Modeled loosely on the widely used Kahn Academy videos (Kahn 2012), the 50 videos produced for Biometry were based on lecture material developed over several previous iterations of the course. Each lesson tended to be shorter than the previous lectures, and was variable in length, freeing the professor to re-structure the presentation of material into more logical units than were possible with the constraint of a 50-minute lecture period (Fig. 1). Using tablet input, the presentation was very similar to what was done in class in prior course offerings. Typically the recordings were done in one ‘take,’ however the format does not preclude re-doing all or part of a video. The format could also allow more formal and scripted presentation of material, such as PowerPoint; however, we have found that teaching mathematical material, such as statistical formulas, using a whiteboard format allows students to follow along and take better notes. Distribution of lengths of video lessons produced for Biometry class. The 10% of lecture material retained for the classroom period revolved around principles of statistics that were illustrated by activities the students were engaged in during class. For example, we illustrated the GIGO principle (garbage in, garbage out) by having the entire class rapidly type in a small data set, and then calculate some simple parameters of the distribution. Invariably GIGO errors had been committed by multiple students (and the professor) in the room. The professor gave a short lecture about the consequences of making such an error, and demonstrated techniques to both detect errors and to avoid making them in the first place. Online quizzes on each video were short (typically 5 questions), and questions were selected randomly for each student from a bank of 8-10 questions so that each student received a unique quiz. Most questions were multiple choice, although true-false, and fill-in-the-blank questions were also used. The intent of the questions was to ensure that students had watched the video and taken notes, and that they understood the basic principles being covered. Many questions tested their understanding of concepts through scenarios. For example, to test their understanding of the purpose of a posteriori tests, a description of a research question was given, along with an outline of the experiment performed, then the student was asked to choose the appropriate test to best address the investigator's question, while controlling for Type I error in a manner that was appropriately liberal or conservative (depending on the desires of the researcher). The quizzes were timed so that students were not able to ‘game’ the system by flipping back and forth between the video and the quiz. The quizzes had a closing date coinciding with class time, such that video viewing and quiz completion occurred before class. In emphasizing the construction of a useful statistical toolbox with many components, all assessment units were relatively low stakes; enough to encourage full participation, but not discouraging continued learning as often happens when one assignment is given a large weight. In total, contributing to the final grade were 25 online quizzes covering the 50 videos (20% of course grade), 7 unannounced quizzes (20% of course grade), 11 homework sets (50% of course grade), and class attendance (10%). During class ‘lecture’ periods, sample problem sets were handed out to give students hands-on practice with real-world bio-statistical problems. Some were given deliberately as individual projects to ensure that each student learned all of the methods and had practiced using the software to perform calculations. More than half were group assignments, with either pairs or groups of four working together to solve problems. The professor briefly showed students the sequence of steps to carry out analyses, then both professor and graduate student teaching assistant circulated among groups as necessary to help coach students when they were stuck. Often groups presented their findings to the entire class in order to illustrate a variety of approaches to present outcomes and results. We assessed student opinions about the flipped classroom approach in mid-semester using an anonymous survey in order to enable course corrections. We found that for 90% of students, this was their first experience with the flipped classroom approach. Most students (76%) watched each video only once, but 81% stopped and replayed portions of the videos often, or every time they watched them. This capability is one of the often-touted advantages of the flipped classroom (Bergmann and Sams 2012), and clearly students do take advantage of this. A disadvantage of remotely-watched lectures is that it is harder to ask questions about the material. Asking a question during a class lecture can catch an instructor error, or highlight a broader need for clarification, or result in addressing a misunderstanding with more immediacy than is possible with a remotely watched lecture. Encouraging students to email questions to the instructor, which can later be answered before the whole class, may be a partial solution to this problem. Despite the very low stakes represented by each short quiz (<1% of the course grade), students freely admitted that having such an assessment on the videos, with a deadline, was important in motivating them to watch the videos (only 14% said it was ‘not important’). Students did feel that the time they were given to complete the video quiz (typically 5 minutes) was too short, so this time was extended (10 minutes) for the remainder of the semester. The students seemed to like the format of the videos, and most preferred seeing the instructors face while narrating (only 5% found it ‘distracting’). Our university uses standardized student evaluations of instruction, with ratings of a common set of questions on a scale of 0 – 5 (corresponding roughly to poor, fair, satisfactory, good, excellent). These were administered in both 2012 (pre-flipped) and 2013 (post-flipped) formats. A total of 19 course evaluation questions were in common between the two years, ranging from questions about the course (e.g., organization, testing, atmosphere, syllabus, grading fairness), about the instructor (e.g., timeliness, mastery, enthusiasm, speech, adherence to syllabus), and summary items (teaching effectiveness, overall course rating, and overall learning). A two-way ANOVA without replication was performed with mean score for each question as the dependent variable; question number and year were the independent variables. Mean ratings of the course and instructor were significantly higher (F = 7.90, P = 0.0064) in 2013 () than in 2012 (). Results for the three summary ratings mentioned above are used to evaluate teaching merit for instructors in our department (Fig. 2). Mean ± SE for summary course ratings of Biometry by undergraduate students. (A) Teaching effectiveness, (B) Overall course rating, (C) Overall learning. In both years of the course, on the first day we performed an incoming assessment of statistics knowledge using questions that were taken from later homework sets. There was a tendency for graduate students in 2012 to have greater incoming knowledge than 2013 graduate students (F = 4.13, P = 0.0650). Therefore, for other performance metrics, we used only undergraduate scores, for which we had a larger sample size, and there was no significant difference in incoming proficiency (F = 0.98, P = 0.3263). Although attendance was incentivized equally in 2012 and 2013, we were concerned that students might be tempted to attend less frequently when lecture material was moved outside the classroom. However, attendance was high in both years and there was no difference between the two years (97% in both). Participation in video watching in 2013, as measured by online quiz-taking, was also high (98%). Of the three major performance-related components of the course grade, the unannounced in-class quizzes best evaluated the state of the statistical toolbox of students. The small sample size and large variance among students prevented us from detecting a mean difference in pop quiz performance. However, the distribution among grade classes (within the A, B, and C groups) did shift (loglikelihood ratio chi-square = 8.19, P = 0.0166; Fig. 3). Specifically, there appeared to be a shift from the B and C groups in 2012 to the B and A groupings in 2013. The most parsimonious explanation is that students who applied themselves were achieving greater working knowledge of statistics with the flipped classroom approach. The proportion of D's remained about the same. The D-level students were likely more disengaged from the class and were not carrying out assignments outside of class; they were therefore unprepared for in-class quizzes with either classroom model. This is supported by the additional fact that every student who missed more than one online quiz received an average of F on pop quizzes. Grades received from undergraduate biology students on unannounced class quizzes as a test of their statistical toolbox. No significant difference was found between homework or final course grades for 2012 and 2013, but this is not readily interpreted since the grading rubric and weightings differed for the two years; 2013 had the added grade element of quizzes on online videos. Most flipped classrooms have students doing hands-on homework or exercises during the “lecture period,” and have students watch lectures outside of class (Bergmann and Sams 2008, 2012, Willey and Gardner 2013). In our implementation of the flipped classroom for Biostatistics, we retained the same number of outside homework assignments as the pre-flipped classroom, but expanded our inclass practice, in addition to moving the traditional lecture portion outside the classroom. Since we were already spending about half of our classroom time on in-class practice, this was not a drastic change, but we believe it did ensure that most students were motivated to do more work in total for the class. At the same time, we retained a small portion of class time for synthesis and ‘lecturing’ on principles of statistics that are best illustrated with ‘live’ demonstrations. For example, in demonstrating the effect of sample size on the chance of making a Type II error (concluding there is no difference among means, when in fact there is), we generate unique data sets for each student to work with, all based on the same parametric mean and effect sizes for groups, but given a random error based on the normal distribution. Starting with the full data set, students show that the entire class can detect significant differences among means with ANOVA, but as N is systematically reduced, more and more students in the class begin committing Type II errors as their analyses show lack of significance. This kind of demonstration provides grist for discussions about how to guard against Type II errors, and the relationship to Type I error. While such lessons can certainly be learned by a student on his/her own, they are vivid, and more memorable, with a population of students. This kind of approach combines the flipped classroom philosophy with the ‘balanced amalgamated’ learning strategy that has been shown to be effective in teaching statistics (Vaugh 2009) Bergmann and Sams (2008, 2012), who are credited with ‘inventing’ the flipped classroom, list 15 reasons why you should use this model for teaching. Having originated in a high school environment, only a subset of these apply to college and university students. Three that resonated in the context of our course were; (1) Flipping helps struggling students; the ability to re-watch parts or all of a lecture may address a real need – in Biostatistics, there are some concepts that may prove difficult to grasp on the first listen. (2) Flipping increases student-teacher interaction; it may not feel like this to the professor, who is talking to the students through the medium of video, but during class periods, there are many more opportunities for one-on-one conversations as students work through problems. (3) Flipping increases student-student interaction, which has been shown to help certain types of learners to retain knowledge better (Johnson et al. 1998). Solving statistical problems in biology is something that seasoned researchers do instinctively because they have wrestled with them many times before in their own research. However, statistical competence is typically hard-won through experience, and difficult to teach in the classroom. Flipping the classroom by moving lecture material to short, focused videos that distill the essence of important course material has the potential to open more class time for both better and more guided learning exercises to give practice with biological/statistical problem solving. Creation of the videos is a non-trivial time investment, both in the planning and production. The additional creation of questions to test student comprehension of the videos appears to be essential to ensure that the videos are watched before class; this counters one of the potential downsides of the flipped classroom – nonparticipation outside of class. The net result of the time students invest in watching videos, taking quizzes, and in-class guided practice with problem-solving does appear to enhance their working knowledge of biostatistics. Other courses that entail numerical or conceptual problem-solving could similarly benefit from such an approach. Strategically deciding when to use the flipped classroom approach, rather than wholesale abandonment of traditional lectures, may be the best approach to enhance student learning in any given course.