Using Low Cost Eye-tracking to Verify Decision Aid (Dis)Use

Abstract

Event Abstract Back to Event Using Low Cost Eye-tracking to Verify Decision Aid (Dis)Use Amanda E. Harwood1*, Carryl L. Baldwin1, Amanda E. Kraft2, Alison M. Perez2, Trevor M. Sands2 and Barlett A. Russell2 1 George Mason University, Human Factors and Applied Cognition, United States 2 Lockheed Martin (United States), United States A current Air Force interest is to design systems that allow one individual to operate several unmanned aerial vehicles (UAVs) at a time. Decision aids that advise the user during operations (e.g. target classification confidence) have the potential to aid performance and decrease workload during these multiple UAV operations, particularly during high risk, high pressure operations. However, in order to be effective UAV operators must calibrate their trust effectively to the capabilities of the decision aid - not becoming complacent (over-trusting) nor under-trusting to the point of disuse. In this study, 42 participants performed a 20-minute multiple UAV simulated operation task using the Supervisory Control Operations User Testbed (SCOUT). The scenario required them to search up to three simulated UAV sensor feeds for a target under either high or low risk conditions. A decision aid suggested whether sensor feed items were targets along with a varying level of confidence in its suggestion. The risk manipulation impacted performance. We used a Gazepoint GP3 eye-tracker to investigate on a trial-by-trial basis how often participants consulted the automated decision aid before making their determination. In the first two and a half minutes participants consulted the decision aid on approximately forty-two percent of trials, declining significantly to approximately ten percent by the last two and a half minutes of the task (see Figure 1). Despite a decrease in consulting, participants performed significantly better as time on task increased from approximately eighty percent accuracy to approximately eighty-eight percent (see Figure 2). Reported mental workload was negatively correlated with decision aid use, meaning that participants who used the decision aid less reported higher mental workload. Participants may have discontinued use of the automated decision aid for several reasons. First, it is possible that the task was manageable without the use of the decision aid. Alternatively, participants may have misunderstood or lacked trust in the decision aid despite a fairly comprehensive training procedure. Regardless of why they discontinued use, verification of use via the eye tracking data provided important information that can be used to guide future work in improving the design of decision aids. Figure Captions: Figure 1. Average percent of participants who checked the decision aid as time on task increased. Figure 2. Average accuracy of participants as time on task increased. Figure 1 Figure 2 Keywords: eye tracking, Automation, Decision aid, Trust, UAV Operations Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Poster Presentation Topic: Neuroergonomics Citation: Harwood AE, Baldwin CL, Kraft AE, Perez AM, Sands TM and Russell BA (2019). Using Low Cost Eye-tracking to Verify Decision Aid (Dis)Use. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00041 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 02 Apr 2018; Published Online: 27 Sep 2019. * Correspondence: Ms. Amanda E Harwood, George Mason University, Human Factors and Applied Cognition, Fairfax, United States, aharwood@gmu.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Amanda E Harwood Carryl L Baldwin Amanda E Kraft Alison M Perez Trevor M Sands Barlett A Russell Google Amanda E Harwood Carryl L Baldwin Amanda E Kraft Alison M Perez Trevor M Sands Barlett A Russell Google Scholar Amanda E Harwood Carryl L Baldwin Amanda E Kraft Alison M Perez Trevor M Sands Barlett A Russell PubMed Amanda E Harwood Carryl L Baldwin Amanda E Kraft Alison M Perez Trevor M Sands Barlett A Russell Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.