Human performance and embedded intelligent technology in safety-critical systems

Abstract

Information technology continues to evolve rapidly. We see this particularly in the evolution of embedded intelligent systems—knowledge-based systems deployed in larger hosts with real-time response requirements, which provide real-time advice, guidance, information, recommendations and explanations to their users. These systems have recently been deployed in safety-critical large-scale systems, where humans and technology are jointly responsible for executing tasks, monitoring operations, and providing system safety. Thus, human interaction with intelligent technology in safety-critical systems has important implications. Those interactions can enhance or reduce system efficiency, enhance or compromise safety, and augment or negate the other benefits that technology provides. In this paper, we focus on interactions between human operators and embedded intelligent systems. We first consider the role of technology in safety-critical systems, and discuss studies of the impact of technology on human operators in such systems. We then describe embedded intelligent systems, and studies of their impacts on human operators. To illustrate these points, we consider the case of embedded intelligent technology introduction in one such setting, and the results of an empirical investigation of the impact of the technology on human performance in that system. We conclude with a discussion of the implications of the study and of the importance of understanding the impact of embedded intelligent technology on human operators in safety-critical systems.