The Translators: The Media and School Choice Research

Abstract

THE MEDIA play a pivotal role in determining how and why research influences opinion with regard to policy. Political scientists Shanto Inyengar and Donald Kinder have shown through experimental research involving televised news how the presentation of news stories can have a powerful impact on what Americans think about issues. (1) Prominent columns and articles, especially in the big East Coast papers, influence political behavior among the policy and political elites and offer signals about elite thought and opinion on key issues. The debates about the research on school choice illustrate the broader challenges the media face when translating research for consumption. At a superficial level, school choice is a relatively easy debate for the media to cover. It can be simplified into arguments for and against vouchers, charter schools, and altering the definition of public schooling, and these arguments are often boiled down to an easy framework of public versus private. Likewise, the question of increases in test scores fits readily into a debate about whether school choice is working or not. While such framing greatly oversimplifies the issues, it nonetheless drives much of the coverage precisely because it offers easy contrasts. Yet research usually offers nuance rather than stark contrasts, and the intersection between school choice research and journalism brings to the surface a key tension between social science and journalism more generally: their different tolerance for and approaches to handling with regard to how definitive findings are. This is not to say that journalists are cavalier about error. On the contrary, most publications employ elaborate fact-checking and editing procedures. But, in addition to its reliance on formal, replicable methods of inquiry to answer questions, social science often parts ways with journalism in its approach to error. There are two kinds of error in social science research: saying something is true when in fact it is false, or saying something is false when in fact it is true. The bias within social science is toward making the latter mistake, known more formally as a Type II error. In other words, when in doubt, favor the non-finding over the finding. Conversely, the natural bias in journalism is toward the Type I error, reaching the conclusion that something is true (publishing the story) even if it later turns out to be false. Put another way, while both fields prize accuracy, journalists are necessarily more concerned with the time-bound nature of news and events and so prize timeliness over certainty. This is not a new story. Richard Colvin, executive director of the Hechinger Institute on Education and the Media, notes that this has always been a source of friction between social scientists and the press. But, he observes, it is more and more prevalent because of growing competition from online media and increasing pressure on news outlets to report news quickly. (2) And it is a healthy tension. Newspaper stories are point-in-time projects, while the accretion of knowledge over time is the process in social science. People read newspapers to find out what is known at present. Research findings, which generally are part of a larger body of evidence and are often not definitive, must be presented in the appropriate context to be truly accurate and useful for readers. And, of course, single studies, regardless of their quality, should be considered cautiously. The conflict arises when journalists seek a definitive angle to build a story around. Too often studies of test scores related to different school choice initiatives provide just such a slant. For instance, in a widely publicized episode, an analysis that offered no basis for causal claims, offered mixed results, and diverged from other research still landed on the front page of the New York Times in 2004 under the headline Nation's Charter Schools Lagging Behind, U. …