Vulnerabilities in Discovery Tech

Abstract

Recent technological advances are changing the litigation landscape, especially in the context of discovery. For nearly two decades, technologies have reinvented document searches in complex litigation, normalizing the use of machine learning algorithms under the umbrella of “Technology Assisted Review” (TAR). But the latest technological developments are placing discovery beyond the reach of attorney understanding and firmly in the realm of computer science and engineering. As lawyers struggle to keep up, a creeping sense of anxiety is spreading in the legal profession about a lack of transparency and the potential for discovery abuse. Judges, attorneys, bar associations, and scholars warn that lawyers need to closely supervise the technical aspects of TAR and avoid the dangers of sabotage, intentional hacking, or abuse. But none of these commentators have defined with precision what the risks entail, furnished a clear outline of potential dangers, or defined the appropriate boundaries of debate. This Article provides the first systematic assessment of the potential for abuse in technology-assisted discovery. The Article offers three contributions. First, our most basic aim is to provide a technical but accessible assessment of vulnerabilities in the TAR process. To do so, we use the latest computer science research to identify and catalogue the different ways that TAR can go awry, either due to intentional abuse or mistakes. Second, with a better understanding of how discovery can be subverted, we then map potential remedies and reassess current debates in a more helpful light. The upshot is that abuse of technology-assisted discovery is possible but can be preventable if the right review processes are in place. Finally, we propose reforms to improve the system in the short and medium term, with an emphasis on improved metrics that can more fully measure the quality of TAR. By exploring the technical background of discovery abuse, the Article demystifies the engineering substrate of modern discovery. Undertaking this study shows that lawyers can safeguard technology-assisted discovery without surrendering professional jurisdiction to engineers.