Decoding object representation using magnetoencephalography

Abstract

Event Abstract Back to Event Decoding object representation using magnetoencephalography Gustavo Sudre1*, Dean Pomerleau2, Mark Palatucci1, Anto Bagic3 and Tom Mitchell1 1 Carnegie Mellon University, United States 2 Intel Labs, United States 3 University of Pittsburgh Medical Center, United States Several studies have used magnetoencephalography to investigate how the human brain processes language and recognizes objects. This work follows the footsteps of the encouraging results obtained by this group using fMRI to predict brain activity associated with the meaning of nouns. Here, we leverage the superior temporal resolution of MEG data to analyze the spatiotemporal activation following the presentation of concrete nouns, and then decode the different nouns presented to the subject during the recording session. More specifically, ten subjects were presented with 20 different questions about properties of 60 different objects. These questions probed properties such as size, material, and usage of each of the 60 objects (e.g. is it made of metal? Is it alive?). Each object was represented by its picture and name (text), displayed simultaneously on the screen. Subjects answered yes or no to the question at hand by pressing the right or the left response pad, respectively. The subjectʼs response also controlled when the current stimuli for a given object disappeared from the screen, giving the subject more control over the experiment and also making the task more engaging. In a separate experiment, the subjects were asked to think about properties of the 60 objects when they appeared in the screen, being as consistent as possible for each repetition of the same experiment. The MEG data were spatially filtered using Maxfilter [1], and then processed using MNE software [2]. Different machine learning techniques were implemented in MATLAB [3] for decoding, and preliminary results show accuracies as high as 91% (mean over subjects: 79%) for classifying between two different words that the classifier has never seen before [4]. These results show that MEG data can be reliably used to decode concrete nouns presented to a subject, and the analysis of the features used for decoding may provide valuable insights to the inner-workings of the brain related to semantic representation.