Estimation of Viewed Image Categories via CCA Using Human Brain Activity

Abstract

This paper presents a method to estimate viewed image categories via canonical correlation analysis (CCA) using human brain activity measured by functional magnetic resonance imaging (fMRI). The proposed method enables estimation of image categories that a subject viewed by using only the subject’s brain activity. Specifically, the proposed method calculates the projection matrices that enable direct comparison between human brain activity and images that subjects viewed through CCA. After projecting the human brain activity and the viewed images on the same latent space, k-Nearest Neighbor (k-NN) is performed to estimate the viewed image categories from only human brain activity. Through the projection matrices, the proposed method can increase training data for k-NN even if a large number of pairs of human brain activity and images cannot be prepared. Experimental results for ten subjects show the effectiveness of the proposed method.