Policy Pilots: Experimentation in the Administration of the Law

Abstract

Continuation of article is available at: https://ssrn.com/abstract=3499202 Rigorous tests are being used every day to develop effective medical treatments, drive consumer engagement, and, more generally, discover what works. But so far, rigorous policy piloting — temporarily introducing a change in law or policy in order to learn from it using well-designed and well-implemented methods — has not been used widely because of the perception that policy experimentation is unfair and possibly illegal, difficult, and rare. This Essay draws upon case law and agency practice to show that, to the contrary, rigorous policy pilots are presumptively legal, feasible, and increasingly common, proceeding in several steps. First, it finds that many kinds of pilots, including those that vary internal agency processes, or which are opt-in are unlikely to be controversial. But a review of relevant cases suggests that courts are likely to uphold even pilots that treat like members of a population differently, including through randomization, when they advance learning. Further, it finds experimentation, by itself, to be unlikely to create special procedural or substantive hurdles. Second, it finds that agencies are engaging in a range of rigorous piloting activities to fill informational gaps in policy- and law- making, some of which simulate and others which effect policy variation on a temporary basis, and that developments such as the growth of open data are making such forms of information gathering easier. It draws from agency experience to develop a framework for proposing a policy pilot and identify steps that would further support the use of rigorous pilots. A companion online appendix applies this framework to propose several rigorous pilots that the United States Patent and Trademark Office (“USPTO”), building on its already strong tradition of piloting, could try to evolve its own policies and practices with respect to patent quality (through the robust vetting of applications in view of non-patent literature and team/time examination on demand) and inclusion in innovation (through automated correction and addressing gender bias in examination). A first pilot would give applicants the option of requesting that all of the relevant art in their case be provided up front, through the examiner's initial searching of the entire specification rather than just the claims, following the approach of jurisdictions that bifurcate search and examination. The early certainty would help applicants make early determinations about whether or not the patent was worth pursuing. Quality pilots, in general, could also more explicitly measure the robustness of art vetting as a quality metric, given the prior art between US examiners and others reviewing a patent. Piloting development of error detection technology for vetting compliance with the Patent Act's Section 112 disclosure requirements, following previous examples of agencies adopting outside technology, could also help close the applicant readiness gap that this essay documents between smaller and larger inventors, enhancing both inclusion and patent quality. Finally, the essay proposes a pilot for testing for the presence of implicit gender bias in the award of patents to help get at the root causes of the 7-21% difference in patent grant rate between male and female-led inventions.