American and Japanese Achievement ... Again A Response to Baker

Abstract

Needless to say, I am grateful for the attention David Baker has given to my article Comparing American and Japanese Achievement: Is the United States Really a Low Achiever? (Westbury, 1992). I acknowledge, too, my appreciation of what he says about the approaches to cross-national analysis that I advocated there. But the analysis he reports leads him to question my claims about the overall and efficiency of the American educational system when compared with that of Japan, at least in the teaching of mathematics at the 7th/8th and 12th grade levels. The issues that circle around this disagreement are large ones that seem to provoke continuing interest (see, e.g., Beardsley, 1992; Cipra, 1992; Stevenson, 1992; Stevenson, Chen, & Lee, 1993; Stevenson & Stigler, 1992), are clearly important, and cannot be resolved by any one argument, analysis, or line of work. My hope for my article was that it might begin just the kind of debate that Baker has joined, and because Baker's analysis rests on the same data set that I work with, the findings he reports and the questions he raises about my analysis become the more compelling. Baker's response questions the reasoning of Comparing American and Japanese Achievement by approaching the analysis of the comparative effectiveness of the two school systems in a very different way. And based on the results of his analysis, Baker argues that I am wrong in claiming that there is little or no evidence in the SIMS data for the widely believed claim of profound systemic differences between the U.S. and Japanese school systems that go beyond differences in the curricula institutionalized in the two countries' school systems. Thus, in contrast to my emphasis on intentional between-country differences in curriculum as the key variable explaining American-Japanese differences in patterns of achievement, Baker finds that when curricular differences are held constant, the learning of U.S. students falls below that of Japanese students; he attributes this failure in American performance to pervasive systemic weaknesses in school system organization, classroom management, and teaching-which, presumably, affect most classrooms (see also Baker & Schaub, in press; Schaub & Baker, 1991). Variables derived from the SIMS Opportunity-to-Learn (OTL), or curriculum coverage, questions lie at the heart of the new evidence Baker's response provides on the role of the curriculum versus other factors in discussions of the relative of U.S. and Japanese schools. He uses OTL to show that Japanese students learn significantly more of the curriculum they are taught than do U.S. students. He follows this up by also showing that achievement gain scores over the school year are, overall, positive in Japanese classrooms and are tightly distributed whereas American gains are widely spread-and sometime negative. He sees both arguments combining to raise fundamental questions about my argument, which he interprets to be that if parts of the American educational system produce similar results to the full Japanese system, then school in the two systems is fundamentally alike. This was not quite my claim but I will put aside that disagreement for the moment. The heart of my argument centered on the observation that the intentions of American and Japanese math curricula differ profoundly and that therefore the match between the curriculum embedded in the SIMS tests and the U.S. curriculum was far from a good one. Thus, Flanders (in press) has analyzed the pattern of coverage of the 180 items of the SIMS Population A item pool in textbooks of the kind used in the 64% of U.S. typical or regular Grade 8 math classes. His findings are presented in Table 1. Overall, Flanders found that only 62 (of the 180) SIMS items were covered in the majority of the six textbooks he examined; only 21% of the SIMS items were in all six textbooks. In