Ethical Implications of Predictive Risk Intelligence

Jiya Tilimbe

doi:10.29297/orbit.v2i2.112

Abstract

This paper presents a case study on the ethical issues that relate to the use of Smart Information Systems (SIS) in predictive risk intelligence. The case study is based on a company that is using SIS to provide predictive risk intelligence in supply chain management (SCM), insurance, finance and sustainability. The paper covers an assessment of how the company recognises ethical concerns related to SIS and the ways it deals with them. Data was collected through a document review and two in-depth semi-structured interviews. Results from the case study indicate that the main ethical concerns with the use of SIS in predictive risk intelligence include protection of the data being used in predicting risk, data privacy and consent from those whose data has been collected from data providers such as social media sites. Also, there are issues relating to the transparency and accountability of processes used in predictive intelligence. The interviews highlighted the issue of bias in using the SIS for making predictions for specific target clients. The last ethical issue was related to trust and accuracy of the predictions of the SIS. In response to these issues,the company has put in place different mechanisms to ensure responsible innovation through what it calls Responsible Data Science. Under Responsible Data Science, the identified ethical issues are addressed by following a code of ethics, engaging with stakeholders and ethics committees.This paper is important because it provides lessons for the responsible implementation of SIS in industry, particularly for start-ups. The paper acknowledges ethical issues with the use of SIS in predictive risk intelligence and suggests that ethics should be a central consideration for companies and individuals developing SIS to create meaningful positive change for society.

Highlights

The use of predictive risk intelligence through the combination of Artificial Intelligence (AI) and Big Data is reaching new horizons, alternatively known as smart information systems (SIS)
Using Prewave as a case study, this paper addresses the research question: how do organisations working with predictive risk intelligence perceive ethical concerns related to Smart Information Systems (SIS) and in what ways do they deal with them? To address this research question, data were collected through a literature review, two semi-structured interviews with participants from Prewave, and a review of their website and the company’s case studies
An example was given by Interviewee 1 in relation to predicting risk for a cosmetics company: ‘if you have a cosmetics company producing shampoo or makeup, and if people are publishing that they get allergic reactions related to that product, that would be a risk that can show up on social media, for example, or in news data....So we focus on early detection, so many other companies do real-time risk monitoring, but we aim to detect the risks even before they happen [...] very quickly in multiple languages’ (Interviewee 1)

Summary

Introduction

The use of predictive risk intelligence through the combination of Artificial Intelligence (AI) and Big Data is reaching new horizons, alternatively known as smart information systems (SIS). Cambridge Analytica used Big Data and advanced ML techniques to provide a full suite of services to enable highly targeted marketing and political campaigning, which raised concerns with regards to the privacy of those whose data had been accessed (Gupta, 2018; Isaak & Hanna, 2018) Another ethical issue with predictive risk intelligence relates to a lack of integrity when designing or using algorithms. According to Wachter et al, (2017), this concern arises because SIS use complex and opaque algorithmic mechanisms that can have many unintended and unexpected effects When it comes to transparency and fairness in the automated decision-making process, such as predictive risk intelligence, users or clients only get a limited idea of why a decision has been made in a certain way, which does not mean the decision is justified or legitimate (Wachter, Mittelstadt, & Floridi, 2017a). Such issues highlight the importance of integrating data quality protocols and high ethical standards to mitigate bias and discrimination when using SIS for predictive intelligence (Hacker, 2018)

A Company using SIS for Predictive Risk Intelligence

Conclusion

Findings

Limitations