Measuring Inconsistency, Indeterminacy, and Error in Adjudication

Abstract

Scholars have long debated whether, and to what extent, law constrains judicial decisions. Because law cannot be objectively measured, it is commonly believed that judicial decisions cannot be empirically evaluated on grounds internal to the practice of law. This article demonstrates that empirical analysis of judicial decisions can nevertheless provide objective, albeit limited, conclusions about subjective criteria for evaluating a system of adjudication. It operationalizes three criteria—inter-judge inconsistency, legal indeterminacy, and judicial error—and clarifies what can be inferred about them from observational data on single-judge adjudication. The precise level of inconsistency cannot be identified, but it is possible to estimate a range of feasible values. Similarly, rates of indeterminacy and error cannot be estimated in isolation, but it is possible to estimate a set of feasible combinations of these rates. The methodologies developed in this article are illustrated using data on immigration adjudication.