Towards a Synthesis of Judicial Perspectives on Technology-Assisted Review

Abstract

DOCUMENT review accounts for an estimated 70% of all e-discovery costs. This means that document review also represents the greatest area of potential cost savings in e-discovery. Over the past ten years, the number of technological approaches to document review for litigation has increased, with the application of methods long in the domain of information retrieval. These advances call upon lawyers practicing civil litigation to gain a familiarity with the various technological options available, lest an opportunity is missed or the opponent, court, or client catches the unwary lawyer by surprise. Technology-assisted review (TAR) is not a luxury available only to large firms handling very large cases. Properly chosen and deployed, technological methods can enable a smaller firm to handle larger cases, and thereby compete with larger firms. In the end, the goal is to make litigation more cost-effective, allowing more cases to stay in the judicial system rather than having litigants settle because the discovery costs outweigh the value of the matter. In addition to gaining an understanding of the technology behind the available techniques, lawyers should appreciate that the match (or mismatch) between the problem to be solved, the technology chosen, and the expertise of the user are as important as the technology in determining whether the results will be both satisfactory and defensible. Part I of this article explains TAR and the variety of approaches available. Part II synthesizes the meaning of the various judicial decisions dealing with TAR in a substantive fashion. Consideration of the judicial opinions proves the importance of understanding available technology. If there is an overall theme from the cases on this topic, it is that courts increasingly expect counsel to consider technological approaches and be competent to discuss what is warranted for a case. While the producing party still has the power to decide how to handle its production, if parties do reach agreement on an approach, they will be held to it, no matter how ill-advised it might turn out to be. Counsel must bring to the conversation sufficient expertise to understand the nuances and import of any proposed discovery protocol applying technology. I. Technology Assisted Review Before 2000, review was done with large teams of document reviewers. Between 2000 and 2010, online review platforms entered the market, hosting whole cases on a single platform and reducing the transactional costs involved in having to manually pull or deliver documents. As court rules and courtroom lawyers began to focus on discovery of electronic information, research turned to understanding how search could be applied to the review process. Research conducted from 2006-2011 under the auspices of the National Institute of Standards and Technology showed that TAR could significantly outperform or significantly underperform human review, depending on the tools used and the expertise of the users. An analysis of some of those results published in the Richmond Journal of Law and Technology (JOLT') concluded: Overall, the myth that exhaustive manual review is the most effective--and therefore, the most defensible --approach to document review is strongly refuted. Technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort. (1) The methods used by participants that returned top results ranged from sophisticated Boolean search to machine learning algorithms; some of the methods that performed less well fell into the same categories. With the help of cases like United States v. O'Keefe and Victor Stanley v. Creative Pipe in 2008, the conversation about the process and expertise needed to successfully navigate document discovery in the increasingly electronic world of both individual and corporate clients entered the legal bar. Those foundational cases, followed by the JOLT article and the article Search, Forward (2) in Law Technology News authored by SDNY Magistrate Judge Andrew Peck, increased the awareness of the uses of search in this context. …