Assessment of Situation Awareness Conflict Risk between Human and AI in Process System Operation

Abstract

The conflict between human and artificial intelligence is a critical issue, which has recently been introduced in Process System Engineering, capturing the observation and action conflicts. Interpretation conflict is another source of potential conflict that can cause serious concern for process safety as it is often perceived as confusion, surprise, or a mistake. It is intangible and associated with situation awareness. However, interpretation conflict has not been studied with the required emphasis. The current work proposes a novel methodology to quantify interpretation conflict probability and risk. The methodology is demonstrated, tested, and validated on a two-phase separator. The results show that interpretation conflict is usually hidden, mixed, or covered by traditional faults, and noises in observation and interpretation, including sensor faults, logic errors, cyberattacks, human mistakes, and misunderstandings, may easily trigger interpretation conflict. The proposed methodology will serve as a mechanism to develop strategies to manage interpretation conflict.