A Plea for More Aggregation: The Looming Threat to Empirical Legal Scholarship

Abstract

Empirical scholarship in the social sciences and the law stands at a critical threshold. As the volume of statistical analysis grows, it becomes increasingly difficult to assess what we actually know about a particular phenomenon. Con-tradictory findings abound, and the social sciences and empirical legal studies currently lack effective tools to filter the good claims from the bad and to synthesize the high-quality findings. This paper explores how to fundamentally change and improve the way these fields generate and use empirical knowledge. I start by examining why the social sciences are poorly equipped to draw empirical conclusions. The root problem is a misguided philosophy of science. Contrary to what social scientists are taught and believe, empirical knowledge does not come from (deductively) testing hypotheses but rather from (inductively) measuring effect sizes. Induction, however, requires a far more holistic perspective than deduction. Given their mistaken philosophical foundation, the social sciences and empirical legal scholarship have undervalued and thus underdeveloped such perspectives. I then argue that the solution is to develop rigorous, evidence-based systematic reviews for observational research. These reviews have been revolutionizing fields like medicine and epidemiology that use randomized clinical trials, but they are almost unheard of in areas that rely on observational work. I demonstrate why these reviews are substantially su-perior to the more informal approaches currently used. I also discuss many of the significant challenges that face those who wish to design such reviews for observational work and point to possible solutions.