'Statistical Dueling' with Unconventional Weapons: What Courts Should Know About Experts in Employment Discrimination Class Actions

Abstract

When evidence is offered in a litigation context, the result can be bad law and bad statistics. Recent high profile, high-stakes employment discrimination class actions bear this out. A series of similar cases litigated over the past several years involve potentially misleading testimony, purporting to show an absence of any pattern of discrimination. Courts in these cases have not always understood the limitations of the evidence before them, or properly weighed its relevance to a ruling on class certification. Because the decision about whether or not to certify a class is critical to both sides, these errors may generate substantial costs. In recent cases against large multinationals like UPS, Wal-Mart, and Marriott, plaintiffs have claimed that decentralized and highly discretionary management practices result in systematic gender or racial disparities in pay and promotion. At class certification, plaintiffs have relied in part on analyses of the company's workforce showing companywide inequality. Defendants have responded with presentations of their own, which frequently demonstrate widely varying outcomes for members of protected groups in different geographic areas or hierarchical levels of the company. These expert submissions usually suggest either that no problems exist, or that any discrimination is isolated and not attributable to institutional-level bias. In adjudicating between these competing visions, courts must referee what the Second Circuit terms statistical dueling. As we show in this paper, sometimes at least one of the parties is dueling with unconventional weapons. For-profit consulting companies and large defense firms are eagerly marketing unorthodox and unreliable methods to employers anxious about how class actions multiply their potential liability. Of course, plaintiffs can, and sometimes do, submit evidence that is inconsistent with good social science practice and biased in favor of their position. However, a bias in the opposite direction is often built into the approach of experts working for defendants, and courts and litigators have largely ignored this bias. Moreover, while the issue as outlined below will be immediately apparent to social statisticians, it has received little attention from either academic statisticians or consulting experts. Using simulated data, we show why courts should become more critical of expertise purporting to test for subunit differences, particularly when offered at the class certification phase of the case. Under some circumstances, the approach often used to oppose class certification in employment discrimination litigation is guaranteed to support the defendant's position, regardless of the actual facts of the case. Furthermore, some courts have improperly or unwittingly legitimized the use of this approach, even when it is demonstrably non-probative of the issues before the court. Courts need new ways to think about these problems - approaches that better reflect the relevant legal framework and principles.