Optimal detection of objects in images and videos using electroencephalography (EEG)

Abstract

The Rapid Serial Visual Presentation (RSVP) protocol for EEG has recently been discovered as a useful tool for highthroughput filtering of images into simple target and nontarget categories [1]. This concept can be extended to the detection of objects and anomalies in images and videos that are of interest to the user (observer) in an applicationspecific context. For example, an image analyst looking for a moving vehicle in wide-area imagery will consider such an object to be target or Item Of Interest (IOI). The ordering of images in the RSVP sequence is expected to have an impact on the detection accuracy. In this paper, we describe an algorithm for learning the RSVP ordering that employs a user interaction step to maximize the detection accuracy while simultaneously minimizing false alarms. With user feedback, the algorithm learns the optimal balance of image distance metrics in order to closely emulate the human's own preference for image order. It then employs the fusion of various perceptual and bio-inspired image metrics to emulate the human's sequencing ability for groups of image chips, which are subsequently used in RSVP trials. Such a method can be employed in human-assisted threat assessment in which the system must scan a wide field of view and report any detections or anomalies to the landscape. In these instances, automated classification methods might fail. We will describe the algorithm and present preliminary results on real-world imagery.