Abstract
We revisit the framework for brain-coupled image search, where the Electroencephalography (EEG) channel under rapid serial visual presentation protocol is used to detect user preferences. Extending previous works on the synergy between content-based image labeling and EEG-based brain-computer interface (BCI), we propose a different perspective on iterative coupling. Previously, the iterations were used to improve the set of EEG-based image labels before propagating them to the unseen images for the final retrieval. In our approach we accumulate the evidence of the true labels for each image in the database through iterations. This is done by propagating the EEG-based labels of the presented images at each iteration to the rest of images in the database. Our results demonstrate a continuous improvement of the labeling performance across iterations despite the moderate EEG-based labeling (AUC <75%). The overall analysis is done in terms of the single-trial EEG decoding performance and the image database reorganization quality. Furthermore, we discuss the EEG-based labeling performance with respect to a search task given the same image database.
Highlights
Successful decoding of brain signals during rapid serial visual presentation (RSVP) triggered the idea of using the electroencephalography (EEG) signals as an extra information channel for image retrieval
EEG-based Image Labeling We report the single-trial EEG classification performance per iteration, averaged across subjects for each search task in the testing phase: ‘‘Eagles’’, ‘‘Tiger’’ and ‘‘Train’’, (Figure 2A)
The performance is given in terms of the area under the curve (AUC) of a receiver operating characteristic (ROC) curve [21]
Summary
Successful decoding of brain signals during rapid serial visual presentation (RSVP) triggered the idea of using the electroencephalography (EEG) signals as an extra information channel for image retrieval. CV techniques encounter a problem known as a semantic gap. It represents the difference between a computational representation of the image and the semantic descriptions that users might employ in any given context. The rationale for EEG-based image search is to link information decoded from brain activity to the semantic description of the presented images. Users can be engaged in the retrieval process by guiding computer vision directly through their EEG channel. In this paper we revisit the framework for brain-coupled image retrieval that relies on the closed-loop synergy between EEG-based image labeling and content-based image retrieval [1]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
