I spy with my AI: The effects of AI-based visual cueing on human operators’ performance and cognitive load in CCTV control rooms

Abstract

The increased number of security cameras in modern cities has elevated the video-feed monitoring demands of closed-circuit television (CCTV) operators. As a result, new AI-driven support systems that leverage the power of computer vision algorithms have been deployed to facilitate the operators' work. However, to effectively design intuitive, AI-driven interfaces and validate their impact on the operators' performance, extensive user testing is required. To address this, we previously developed and tested a virtual reality (VR) control room that can be used to iteratively evaluate intelligent computer assistants and interfaces while operators are subjected to different cognitive load. In the present study, we use this VR environment and physiological markers (e.g., eye tracking measures) to investigate how AI-based visual cueing (i.e., pushing forward video streams on which detections are highlighted by rectangles drawn around targets) affects operator performance and cognitive load. Results suggest that support systems using such technology in a control room improve operators’ performance and decrease their cognitive load, as reflected by changes in pupil dilation and subjective reports irrespective of induced cognitive load.