Figuring out what works in education.

Abstract

In his article “Meager evaluations make it hard to find out what works,” J. Mervis writes about the difficulties of assessing education research (News of the Week, 11 June, p. [1583][1]). The only way to establish a curriculum's effectiveness is through randomized, control trials similar to those used in clinical research. The idea of controlled trials in evaluation is certainly not new, and I learned about them in the 1970s while studying for a Ph.D. in education. Unfortunately, interest in educational program evaluation has remained theoretical in schools. In fact, in frustration, I left education studies and moved to academic medicine, where effectiveness research and systematic literature reviews to find best practices are the norm ([1][2]). I take issue with the idea that controlled studies are difficult to do because “[i]t's hard to exclude people from a program that they think is working.” Controlled studies are based on rational inclusion and exclusion criteria and aim to demonstrate under which circumstances and for whom programs work best (if they work at all). Also, although it is true that measuring human behavior is “hard to quantify,” that does not mean we cannot do it reasonably well. In academic medicine, we have found ways to measure concepts like satisfaction, decision-making, and quality of life. Surely, with effort, educational evaluators can identify valid measures of educational outcomes. One possible explanation for the poor studies we find in education is that evaluations cost as much as program development. The best ones are conducted by multidisciplinary teams of educational researchers and experts in design, measurement, and data analysis. Educational programs are always in a cash crisis. Yet the consequences of implementing programs of unknown quality are likely to be even more costly in terms of the educational opportunity they waste. 1. 1.[↵][3] 1. B. Stein 2. et al. , JAMA 290, 603 (2003) See, for example. [OpenUrl][4][CrossRef][5][PubMed][6][Web of Science][7] [1]: /lookup/doi/10.1126/science.304.5677.1583 [2]: #ref-1 [3]: #xref-ref-1-1 View reference 1. in text [4]: {openurl}?query=rft.jtitle%253DJAMA%26rft.stitle%253DJAMA%26rft.issn%253D0002-9955%26rft.aulast%253DStein%26rft.auinit1%253DB.%2BD.%26rft.volume%253D290%26rft.issue%253D5%26rft.spage%253D603%26rft.epage%253D611%26rft.atitle%253DA%2BMental%2BHealth%2BIntervention%2Bfor%2BSchoolchildren%2BExposed%2Bto%2BViolence%253A%2BA%2BRandomized%2BControlled%2BTrial%26rft_id%253Dinfo%253Adoi%252F10.1001%252Fjama.290.5.603%26rft_id%253Dinfo%253Apmid%252F12902363%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [5]: /lookup/external-ref?access_num=10.1001/jama.290.5.603&link_type=DOI [6]: /lookup/external-ref?access_num=12902363&link_type=MED&atom=%2Fsci%2F305%2F5691%2F1715.1.atom [7]: /lookup/external-ref?access_num=000184558500014&link_type=ISI