Speculation and Conjecture

Abstract

Mr. Krashen disputes the analysis of Richard Innes regarding the impact of whole-language instruction on California reading scores. IN HIS response to my June 2002 article, Richard Innes presents two sets of criticisms, one related to CAP scores and one to NAEP scores. CAP Scores Innes maintains that CAP scores showed an improvement before was introduced in California and a decline after was introduced. The improvement he refers to is the increase in CAP scores for third-graders from 268 to 282 between 1984 and 1987. This, Innes maintains, contradicts the claims that there is no clear pattern in CAP scores and that California's reading problem existed well before was allegedly introduced. The decline he refers to is the drop in scores from 282 to 275 between 1988 and 1990. According to Innes, this change reflects the impact of language. The problem with this analysis is that the recommendations of the 1987 Language Arts Framework were not implemented until much later. California districts did not have textbooks that were consistent with the 1987 framework until the 1989-90 school year,1 which means that the CAP scores presented in my original article were only minimally influenced by the framework committee. If whole language (actually literature-based instruction) had been solidly in place throughout California in the fall of 1989 those in grade 3 in 1990 would have had grade 2 and part of grade 3 under the new literature-based program, and those in grade 3 in 1989 would have had only part of grade 3 under the new program. What caused the increase in third-grade scores that Innes mentions? A look at the entire range of California CAP scores shows that they are part of a general increase that began when the CAP was introduced in California in 1979, not counting a 10-point drop in 1984-85 in grade 8 (Table 1). The increase is real, but two facts must be noted. 1. The increases are not large. The scores presented are scaled scores, with a range of 100 to 400. They are much smaller than percentiles. (The state average is considered to be 250. This is by definition the score that is achieved the first time the test is given.) 2. One expects gains in standardized tests. Research has shown that after new tests are introduced, test scores rise, which is why commercial tests need to be recalibrated every few years.2 Typical test-score inflation for third-grade reading scores ranges from .3 to 1.9 national percentile ranks per year,3 which is about what one sees in Table 1. (John Guthrie and his colleagues claim that this is not the cause of the increases seen in California. They point out that the CAP test does not consist of a small number of items, but is a matrix-sample type of test in which each individual student takes only a small subset of the total items; items vary for each student in the same classroom. This makes the test more resistant to cheating and direct teaching to the items in the test.4 Nevertheless, increasing familiarity with the test format and the kind of questions asked could contribute to test-score inflation.) What about the decline from 1988 to 1990? A look at Table 1 shows that this decline was present in grade 6 as well, making it even less likely that it was due to language, an approach generally used in the lower grades. If we are looking for candidates for the decline, they are easy to find.5 1. The number of children with limited English proficiency (LEP) in California rose dramatically after 1987. From 1984 to 1987 the number of LEP children rose from about 500,000 to 600,000, but from 1987 to 1990 it rose to 850,000, increasing to 1,150,000 in 1992. Thus, while the number of LEP students had been increasing before 1987, the rate jumped sharply after 1987. In California, a large percentage of those taking the NAEP have limited English proficiency, which has a clear effect on scores. …