Do Attorney Surveys Measure Judicial Performance or Respondent Ideology? Evidence from Online Evaluations

Abstract

Which judges are “good” at their jobs, and which are not? The answer may depend on the ideology of whom you ask. Judicial decisions inevitably involve policy making, and lawyers may prefer judges whose policy preferences match their own. This paper tests that prediction with online evaluations of judges. Criminal defense attorneys, a group likely to hold progressive views, make up a disproportionate share of the respondents. The respondents assign lower average scores to Republican appointees, especially female and minority ones, even after controlling for the judges’ backgrounds and performance measures. In comments, respondents object to judges with conservative tendencies more often than those with liberal ones. The objections to conservative tendencies correlate with large reductions in a judge’s numerical ratings, while objections to liberal ones do not. The results suggest that judicial evaluation surveys should take account of how attorneys’ ideology influences their perceptions of judicial performance.