Abstract
Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspiration for machine learning, little effort has been made to directly use data collected from working brains as a guide for machine learning algorithms. Here we demonstrate a new paradigm of “neurally-weighted” machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. We show that our neural-weighting approach can lead to large performance gains when used with traditional machine vision features, as well as to significant improvements with already high-performing convolutional neural network features. The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data.
Highlights
Machine learning is a field of computer science that builds algorithms that learn
Our strategy is to bias the solution of a machine learning algorithm so that it more closely matches the internal representations found in visual cortex
From this set of voxels, 3,569 were labeled as being part of one of thirteen visual regions of interest (ROIs), including those in the early visual cortex. Seven of these regions were associated with higher-level visual processing; all seven higher-level ROIs were used in object category classification tasks probing the semantic understanding of visual information: extrastriate body area (EBA), fusiform face area (FFA), lateral occipital cortex (LO), occipital face area (OFA), parahippocampal place area (PPA), retrosplenial cortex (RSC), transverse occipital sulcus (TOS). 1,427 voxels belonged to these regions
Summary
Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. We demonstrate a new paradigm of “neurally-weighted” machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. Recent years have seen a renaissance in machine learning and machine vision, led by neural network algorithms that achieve impressive performance on a variety of challenging object recognition and image understanding tasks[1,2,3]. This work builds on previous machine learning approaches that weight training[8,37], but here we propose to do such weighting using a separate stream of data, derived from human brain activity
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
