Abstract
Objective. Several neuroimaging studies have demonstrated that the ventral temporal cortex contains specialized regions that process visual stimuli. This study investigated the spatial and temporal dynamics of electrocorticographic (ECoG) responses to different types and colors of visual stimulation that were presented to four human participants, and demonstrated a real-time decoder that detects and discriminates responses to untrained natural images. Approach. ECoG signals from the participants were recorded while they were shown colored and greyscale versions of seven types of visual stimuli (images of faces, objects, bodies, line drawings, digits, and kanji and hiragana characters), resulting in 14 classes for discrimination (experiment I). Additionally, a real-time system asynchronously classified ECoG responses to faces, kanji and black screens presented via a monitor (experiment II), or to natural scenes (i.e. the face of an experimenter, natural images of faces and kanji, and a mirror) (experiment III). Outcome measures in all experiments included the discrimination performance across types based on broadband γ activity. Main results. Experiment I demonstrated an offline classification accuracy of 72.9% when discriminating among the seven types (without color separation). Further discrimination of grey versus colored images reached an accuracy of 67.1%. Discriminating all colors and types (14 classes) yielded an accuracy of 52.1%. In experiment II and III, the real-time decoder correctly detected 73.7% responses to face, kanji and black computer stimuli and 74.8% responses to presented natural scenes. Significance. Seven different types and their color information (either grey or color) could be detected and discriminated using broadband γ activity. Discrimination performance maximized for combined spatial-temporal information. The discrimination of stimulus color information provided the first ECoG-based evidence for color-related population-level cortical broadband γ responses in humans. Stimulus categories can be detected by their ECoG responses in real time within 500 ms with respect to stimulus onset.
Highlights
Real-time detection and discrimination of visual perception could lead to improved human-computer interfaces, and may provide the foundations for new communication tools for people with serious neurological disorders such as amyotrophic lateral sclerosis (ALS).Substantial research based primarily on functional magn etic resonance imaging has shown that categorization of visual perception is implemented by the brain across different regions on the ventral temporal cortex
This study investigated the spatial and temporal dynamics of electrocorticographic (ECoG) responses to different types and colors of visual stimulation that were presented to four human participants, and demonstrated a real-time decoder that detects and discriminates responses to untrained natural images
Real-time detection and discrimination of visually perceived natural scenes is even possible when the system is trained on different data than was presented on a computer screen
Summary
Real-time detection and discrimination of visual perception could lead to improved human-computer interfaces, and may provide the foundations for new communication tools for people with serious neurological disorders such as amyotrophic lateral sclerosis (ALS).Substantial research based primarily on functional magn etic resonance imaging (fMRI) has shown that categorization of visual perception is implemented by the brain across different regions on the ventral temporal cortex. The neural basis of face perception has been investigated with electrocorticographic (ECoG) recordings. Initial work in this area investigated ECoG evoked responses to faces versus scrambled faces (Allison et al 1994), faces versus nonfaces (Allison et al 1999), and more diverse stimuli including faces versus parts of faces versus scaled and rotated faces (McCarthy et al 1999) and faces versus bodies (Engell and McCarthy 2014a). More recent studies of visual perception investigated ECoG broadband responses to faces and other objects (Lachaux et al 2005, Tsuchiya et al 2008, Engell and McCarthy 2011, Engell and McCarthy 2014a, 2014b, Ghuman et al 2014), and used them to predict the N200 evoked response (Engell and McCarthy 2011), or to predict the onset and identity of visual stimuli (Miller et al 2016)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
