A value-sensitive design approach to minimize value tensions in software-based risk-assessment instruments

Abstract

ABSTRACT Many authorities currently use software-based risk-assessment instruments (SBRAIs) to assess a person’s potential to commit crimes (e.g. terrorism, sexual assault). SBRAIs promise better, faster, and more consistent decision-making than their paper-based predecessors. The underlying SBRAI classification schema is well-understood, but given the potentially serious ethical consequences of misjudgments, there is a lack of understanding of how a newly introduced IT artefact, or digital transformation, affects the overall process. Applying a value-centric lens and using Value Sensitive Design (VSD), we identify users’ most relevant SBRAI values, as well as the existing value tensions between developers and users. We find that various value tensions, which had not been an issue prior to digitization, occur. These are tensions regarding the values traceability, reliability, neutrality, support, and trust. These value tensions often emerge due to misunderstandings and miscommunications that unclear classifications enable. Practitioners should therefore focus on four design guidelines to avoid SBRAI value tensions: constant and accessible information, transparency, isolated use anticipation, and compliance with novice and expert users’ requirements.