Response: To Replicate or Not to Replicate: Is That Schuman's Question?

Abstract

U chuman raises a series of particulars concerning our investigation of students' study hours and grades, particulars that boil down to one central issue: Who has a better framework for examining this issue-Schuman, Walsh, Olson, and Etheridge (1 985) or Rau and Durand (2000)? He wonders why we (Rau and Durand) did not more closely replicate their study and why we altered without explanation some of their key questions. After careful consideration (Rau and Durand 2000:21-25), we found two strong reasons why we should not copy Schuman et al.'s design: (1) key questions on study hours contained serious flaws and (2) the research is atheoretical (Mills 1 959:55-75). Flawed questions compromised three of Schuman et al.'s investigations and the minor investigations for which Schuman et al. relied on leading questions to estimate students' study hours. As a result, they could not rule out acquiescence bias as a major confound. Table 1 gives the questions on study hours-questions that make for a good handout in an undergraduate methods course during lessons on the design of questionnaires. Most students in such a course quickly identify several problems. First, these questions assume that students are, in fact, studying and that the pertinent issue is the number of hours devoted to study. Since students who do not study typically feel guilty about their indolence, questions of this sort can pressure them into giving fallacious numbers. Throwing in one weak qualifier (if any) is no solution to acquiescence bias. One has to assume that some students hardly ever study and then to write questions that make them feel comfortable in giving a nonstudy response. That is what we did and why we found that a quarter of Illinois State students do not study during a normal academic week (Rau and Durand 2000:21-22, 35), the same percentage that Moffatt (1 989:293) found at Rutgers University. Unfortunately, we do not know the percentage of nonstudy students at Michigan because Schuman et al. did not allow such students a voice in their surveys. Second, Question 27 arbitrarily determines students' weekends to begin on Friday afternoon and to end on Sunday evening-an unworldly hypothetical, since many students have classes or can be found in the library on Friday afternoon. In contrast, we asked students to tell us when their weekends began and ended: Only 14 percent fit Schuman's procrustean bed. Instead, 38 percent reported that they began their weekends before Friday afternoon, and another 33 percent said they began them on Friday evening or later. In a similar vein, only 41 percent said their weekends ended Sunday evening, compared to 45 percent who said that their weekends ended before then. For some students, Sunday morning is the beginning of their academic week; others do not touch a book until Monday, if then. Third, students' avid commitment to flextime raises a host of questions on