Reexamining Relative Bar Performance as a Function of Non-Linearity, Heteroscedasticity, and a New Independent Variable

Abstract

One might think that if a law school’s graduates do better on the bar exam than their credentials on entering law school would have predicted, the law school’s faculty must have done a good job. Indeed, some scholarship makes that claim. But it is empirically untrue. What actually yields such superficially impressive results is not pedagogy but rather prestidigitation. When law schools manipulate their matriculant pools via academic attrition and transfer, that sleight of hand improves their bar performance rates. This article reveals the math behind the magic. We challenge the assertion that effective pedagogy is the primary driver of a school’s overperformance on the bar examination by demonstrating the model misspecification of the linear regressions in support of that assertion. Our analyses, based on original code written for the program Mathematica, generate regression equations that clearly illustrate the error of applying linear regression to heteroscedastic, non-linear data, and the bias in favor of schools with mid-range LSAT and UGPA matriculants this misapplication causes. Over and under performance on the bar examination, relative to the entering credentials of a school’s matriculants, can be explained in large part by combining the institutions’ academic attrition and net transfer rates into a single independent variable. We demonstrate the marked difference in the value of this variable between schools identified as overperforming and those identified as underperforming their matriculation credentials on the bar examination. The article concludes by emphasizing the harm to legal education that stems from the unwarranted attribution of bar success to pedagogy and especially the harm this causes to untenured academic support faculty. We also raise the question of whether ABA standard 316-mandating a certain level of bar passage by law schools’ graduates-is ethically and morally supportable given the ability of schools to manipulate bar passage rates by modulating academic attrition and transfer rates.