The Effect of Agent Reasoning Transparency on Complacent Behavior: An Analysis of Eye Movements and Response Performance

Abstract

We examined how varying the transparency of agent reasoning affected complacent behavior, in the form of incorrect acceptances of an agent’s recommendations, in a route selection task. We were particularly interested in how participants’ eye movements might disambiguate whether the incorrect acceptances were due to complacency or incorrect information processing. Participants guided a threevehicle convoy safely through a simulated environment of which they had a limited amount of information, while maintaining communication with command and monitoring their surroundings for threats. The intelligent route-planning agent assessed potential threats and suggested changes to the convoy route as needed. Each participant was assigned to one of three agent reasoning transparency conditions. While access to agent reasoning did appear to reduce complacent behavior in one condition, performance in the other conditions indicated potential complacent behavior. An area of interest analysis, reviewed in conjunction with the performance data, indicated the reason behind the participants’ behavior was different between these two conditions. While in the non-transparent condition participants were likely engaging in complacent behavior, in the highly transparent condition it is more likely they were overwhelmed by the amount and/or type of information, resulting in difficulty assimilating the information to support their decision-making task.