The Danger of Rigged Algorithms: Evidence from Immigration Detention Decisions

Abstract

This article illustrates a simple risk of algorithmic risk assessment tools: rigging. In 2017, U.S. Immigration and Customs Enforcement removed the “release” recommendation from the algorithmic tool that helped officers decide whom to detain and whom to release. After the change, the tool only recommended detention or referred cases to a supervisor. Taking advantage of the suddenness of this change, I use a fuzzy regression discontinuity design to show that the change reduced actual release decisions by about half, from around 10% to around 5% of all decisions. Officers continued to follow the tool’s detention recommendations at almost the same rate even after the tool stopped recommending release, and when officers deviated from the tool’s recommendation to order release, supervisors became more likely to overrule their decisions. Although algorithmic tools offer the possibility of reducing the use of detention, they can also be rigged to increase it.