The report of the National Reading Panel is touted as being based on a medical model of research. Offering a perspective from the field of medicine itself, Dr. Strauss examines the report and finds that it falls short on several of the most basic standards of good medical research. By Steven L. Strauss, M.D. THE BEHAVIOR of the National Institute of Child Health and Human Development (NICHD) with regard to the production and dissemination of the report of its National Reading Panel is nothing short of scandalous. In fact, the scandal is of such enormity that it demands a full explanation. The chief element of the scandal is the falsification of the panel's findings. This is immediately apparent when the panel's full report is compared with the short summary version offered for wide distribution. The full report acknowledges that there is scant evidence to justify the claim that phonics instruction leads to improved reading ability, noting that only some normal first-graders showed some gains. But the summary report falsifies the panel's own findings and asserts that phonics instruction correlates with improved reading in a number of areas all the way up to grade 6. Such distortions have been amply documented by Elaine Garan in her important new book and in these pages.1 Of course, it is the summary report that has been distributed to the media and to policy makers. Therefore, it should not be completely surprising to discover that the summary report was prepared, in part, by Widmeyer Baker, a public relations firm. This means that we can, quite accurately, think of the summary report as PR, which certainly puts it in a different light. But it is more disturbing that the PR firm that contributed to the report is the same one employed by McGraw- Hill, a company with a substantial financial stake in the matter of the role of phonics in reading instruction. But the scandal does not stop there. According to Garan -- citing panel member Joanne Yatvin's minority report for the NRP -- the sole reviewer of the phonics section of the NRP report was Barbara Foorman, a NICHD- funded researcher. And of the 38 articles included in the NRP's phonics meta-analysis, Foorman was the lead or secondary author of four. In other words, she contributed more than 10% of the articles reviewed by the NRP and was a reviewer of her own work. The NRP meta-analysis was based on its version of a medical model of research that compares performance criteria before and after So, for example, just as a clinical researcher might compare pain level or muscle strength before and after treatment with drug X, so too did the studies reviewed by the NRP compare reading ability before and after phonics treatment. A recent series in the British Medical Journal reviewed the methodology behind an acceptable meta-analysis.2 The authors specifically noted the importance of a thoroughly blind review of the studies, to the point of suggesting that articles be typographically reformatted so that the reviewers cannot identify the journal, authors, funding, or any other feature that could influence the review process. The authors of the British Medical Journal series pointed out that the quality of meta- analyses is significantly improved by such blinded methodology. The relevance of these concerns to Foorman's review is clear. Foorman's NRP phonics studies are also open to serious criticism on their own terms. In at least two of the studies, Foorman refers to a set of 60 stimulus words that were used to test young readers before and after phonics instruction. Some of the words are labeled regular, and others are labeled exceptional. But no explanation is offered of how the stimulus words were assigned to one category or the other. Fundamentally, this oversight means that there is no way to tell for certain what Foorman means by phonics. Suppose Foorman's understanding of phonics is not the same as that assumed in the other studies included in the meta-analysis. …
Read full abstract
Sep 1, 2003
David B Troxel 
Open Access