Program Design, Incentives, and Response: Evidence from Educational Interventions

Abstract

1. INTRODUCTION Concerns that U.S. students are not performing as well as their counterparts in other developed countries on international math and science tests have led to widespread demands for the reform of K-12 education in the United States. Of the various reforms under consideration, school voucher reform is at the forefront. Vouchers are scholarships that make students eligible to transfer from public to private schools. A basic feature of all publicly funded voucher programs in the United States is the funding of vouchers by public school revenue, so that money always follows students. In other words, schools that lose students lose their corresponding funding. Schools therefore recognize the financial implications of vouchers and have an incentive to avoid being subject to voucher programs. This article investigates the role of program design in the context of two such educational interventions in the United States--the Milwaukee and Florida school voucher programs--and analyzes the effects of design on public school incentives and performance. (1) We demonstrate that variations in program design have markedly different outcomes for public schools affected by vouchers. The Milwaukee program, introduced in 1990, was the first voucher program in the country. Implemented in 1999, the Florida program was the nation's third, following Cleveland's. The Milwaukee and Florida voucher programs share the basic feature of funding by public school revenue. But there are crucial differences. Milwaukee's is a means-tested program targeting low-income students while Florida's embeds a voucher program in a full-fledged accountability system. Using test-score data from Milwaukee and Florida and implementing a difference-in-differences estimation strategy, our study estimates the impact of each program by comparing the post-program results of the affected schools with a comparable set of control schools. Controlling for potentially confounding pre-program time trends and post-program common shocks, we find that the performance effects of the Florida program far exceed those of Milwaukee's program. These results are quite robust in that they hold after controlling for other confounding factors, such as mean reversion and a possible stigma effect; they also withstand several sensitivity tests. Our findings have important policy implications, which we consider in the context of New York State's federal, state, and city accountability programs. These programs include New York City's accountability policy, known as the Progress Report policy, and the federal No Child Left Behind (NCLB) law, as implemented by New York State. Our study is organized as follows. Section 2 describes the Milwaukee and Florida voucher programs. In Section 3, we discuss the incentives created by the programs and the corresponding responses that might be expected from the affected public schools. Our data and empirical strategy are reviewed in Sections 4 and 5, respectively. Section 6 presents our results, and Section 7 considers policy implications. 2. INSTITUTIONAL DETAILS The first publicly funded school voucher program in the United States was established in Milwaukee, Wisconsin, in 1990. The Milwaukee Parental Choice Program made the city's entire low-income public school population eligible for vouchers. Specifically, starting in the 1990-91 school year, the program made all Milwaukee public school students with family income at or below 175 percent of the poverty line eligible for vouchers to attend nonsectarian private schools. In contrast, the Florida Opportunity Scholarship Program, introduced in 1999, can be looked upon as a threat-of-voucher program. Here, failing public schools were threatened with the imposition of vouchers, with vouchers implemented only if schools failed to meet a government-designated cutoff quality level. The institutional details of the Milwaukee and Florida programs are summarized in Table 1. …