Abstract
This piece endeavors to provide context for state and local officials considering tasks around development, procurement, implementation, and use of risk assessment tools. It begins with brief case studies of four states that adopted (or attempted to adopt) such tools early on and describes their experiences. It then draws lessons from these case studies and suggests some questions that procurement officials should ask of themselves, their colleagues who call for the acquisition and implementation of tools, and the developers who create them. This paper concludes by examining existing frameworks for technological and algorithmic fairness. The authors offer a framework of four questions that government procurers should be asking at the point of adopting RA tools. That framework draws from the experiences of the states we study and offers a way to think about accuracy (i.e., the RA tool’s ability to accurately predict recidivism), fairness (i.e., the extent to which an RA tool treats all defendants fairly, without exhibiting racial bias or discrimination), interpretability (the extent to which an RA tool can be interpreted by criminal justice officials and stakeholders, including judges, lawyers, and defendants), and operability (the extent to which an RA tool can be administered by officers within police, pretrial services, and corrections).
Highlights
Across the United States and around the world, local governments are procuring or developing applications known as “risk assessment tools” or “actuarial risk assessments” to aid decision-making in criminal courts
Informed and responsible procurement of RAs requires: a robust understanding of the science behind algorithmic decision-making and the technical, business, legal, and other considerations that drive private developers in this space; sustained engagement and consultation with relevant communities who are impacted by the use of these tools; development of robust frameworks for post-procurement training and guidance on implementation; and deep and sustained commitment to regular evaluation and collaboration with experts and the wider community around assessment of efficacy and bias
A recently-introduced criminal justice reform bill in the California Senate spells out in more detail the factors that a pretrial RA tool should exclude, including education, employment, and housing status, and suggests regularly validating such tools and minimizing economic and racial disparities that may be embedded in criminal history.[49]
Summary
Across the United States and around the world, local governments are procuring or developing applications known as “risk assessment tools” or “actuarial risk assessments” (collectively, “RAs”) to aid decision-making in criminal courts. A robust understanding of the science behind algorithmic decision-making and the technical, business, legal, and other considerations that drive private developers in this space; sustained engagement and consultation with relevant communities who are impacted by the use of these tools ( minority communities who are often overrepresented in the criminal justice system); development of robust frameworks for post-procurement training and guidance on implementation; and deep and sustained commitment to regular evaluation and collaboration with experts and the wider community around assessment of efficacy and bias. Assessing The Assessments | Lessons From Early State Experiences In The Procurement And Implementation Of Risk Assessment Tools This piece endeavors to provide context for state and local officials considering tasks around development, procurement, implementation, and use of RA tools. It begins in Part II with brief case studies of four states that adopted (or attempted to adopt) RA tools early on and describes their experiences.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
