Toward a Model of Responsibility for Proactive Systems

Abstract

1. IntroductionThere is a growing trend to allow computing systems to influence decisions in all fields from medicine to finance. Computers are able to process vast amounts of data in almost real time, allowing them to function much more effectively than a human counterpart. Humans use expert systems, data mining, and the like to suggest possible actions and decisions based on the data that the system evaluates.With the advent of wireless connectivity, more and more computer systems are being linked and are enabled to communicate and share data as needed. Common household electronics now often contain processors, and computing devices are becoming pervasive in our lives. Concepts such as pervasive computing, ambient computing, and proactive computing are at the forefront of research today, with the goal of removing the burden of technical decision-making from the user. Such systems are envisioned as interacting with users in everyday life, often in such a way that the user is not even aware of the interaction. Furthermore, proactive systems will be able to interact physically with users and their environment via sensors and actuators. They will learn from user behavior and adapt to it, and make decisions regarding the environment and users in real time without human supervision to determine the actions of the system. This gives rise to a (Matthias, 2004) between the responsibility of system designers and the responsibility of the system itself. Once the system is operational, it will adapt itself to the situation, in which case it becomes hard to hold a designer responsible if the system makes an incorrect decision.The purpose of this paper is to look at this responsibility gap as it pertains to proactive systems and to suggest a model of responsibility that can be applied to such systems. The model focuses on the practical assignment of responsibility for harmful events caused by autonomous systems, without attempting to place moral responsibility on computers, which cannot be moral agents. We redefine the concepts of prospective and retrospective responsibility (Vedder, 2001) in terms of proactive systems. Section 2 gives a background of proactive computing. Section 3 is a discussion of responsibility as seen in view of normal software projects. Section 4 applies Matthias's to proactive systems and suggests a model of responsibility for such systems. Section 5 concludes the paper.2. BackgroundComputing devices are present in many common household items, cellular phones, personal digital assistants, personal computers and the like. Processors currently outnumber the people in many households. With such an overwhelming number of computers around us, the concept of pervasive computing is fast becoming a reality. Concepts such as pervasive (or ubiquitous) computing (Undercoffer et al., 2003), ambient computing (Jacquet et al., 2005) and proactive computing (Want et al., 2003) are often mentioned interchangeably, and they are in fact closely related. For the purpose of this paper, we make the following distinctions (see Figure 1, page 80). Pervasive computing is the presence of computing devices embedded in everyday items. Ambi- ent computing is the subtle linking of such devices, so that users are not even aware of them while they fulfill their duties. Proactive computing takes the duties to a higher level where a software system that is running autonomously, controls such linked devices, making decisions on behalf of humans, based on data collected by sensors and other devices connected to the network.For the purpose of this paper, the focus falls on proactive rather than pervasive or ambient computing, because the other two paradigms still need humans in the control loop, and the responsibility for actions still rests with human operators or supervisors. Proactive computing, however, is envisioned ultimately to remove humans from the control loop, creating systems that can act autonomously-thus resulting in a question regarding where the responsibility for wrong decisions lies. …