Multisensory Integration via Density Estimation

Abstract

Event Abstract Back to Event Multisensory Integration via Density Estimation Joseph G. Makin1*, Matthew R. Fellows1 and Philip Sabes1 1 University of California at San Francisco, Department of Physiology and Keck Center for Integrative Neuroscience, United States Multisensory integration is the task of combining redundant (i.e., statistically dependent) environmental cues into a single estimate for a common underlying stimulus; for example, the redundant---but presumably noisy---information about hand position provided by vision and proprioception. Statistically optimal combination of these signals weights them by their precisions (inverse variances) and yields a minimally varying estimator of the stimulus. Human perceptual and sensorimotor behavior have been shown to approach such statistical optimality in a variety of contexts, even when the redundant sensory signals are represented quite differently across modalities. For example, when planning and executing a reaching movement, we appear to approximate maximum-likelihood integration of visual and proprioceptive signals of the arm, despite their disparate encoding and the nonlinear relationship between the spaces of the two signals. Given the complexity of this and similar mappings, it seems likely that the neural mechanisms that implement sensory integration also learn it from experience; and indeed, learning of inter-sensory maps has been shown experimentally, in e.g. the auditory-visual maps of the barn owl. Here we ask how more complex multidimensional maps can be learned de novo by a relatively simple network from the joint statistical properties of the inputs. Our approach is to learn to integrate by extracting the underlying causes from the data, via density estimation in a restricted Boltzmann machine (RBM). We show that the model can: learn to integrate nearly optimally; learn prior distributions over stimuli; integrate additional cues in hierarchical stages; learn to combine two independent cues with a third (as with gaze angle, retinal position, and proprioception); and generate missing data, e.g., to make predictions about one modality based on another, or to plan motor actions based on their hypothesized relationship. Acknowledgements Defense Advanced Research Projects Agency Reorganization and Plasticity to Accelerate Injury Recovery (N66001-10-C-2010). References Barlow, H. B. (1961). "Possible principles underlying the transformation of sensory messages," in Sensory Communication, ed. WA Rosenblith. (Cambridge, MA: MIT Press), 217–234. Hinton, G. E. (2002) Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation 14, 1771–1800. Hinton, G. E., Osindero, S., and Teh, Y. (2006) A fast learning algorithm for deep belief net. Neural Computation 18, 1527–1554. Van Beers, R., Sittig, A., and van Der Gon, J. J. D. (1999) Integration of proprioceptive and visual position-information: An experimentally supported model. Journal of Neurophysiology 8, 1355–1364. Keywords: Bayesian optimality, density estimation, machine learning, multisensory integration, Restricted Boltzmann Machine Conference: BC11 : Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meeting 2011, Freiburg, Germany, 4 Oct - 6 Oct, 2011. Presentation Type: Poster Topic: sensory processing (please use "sensory processing" as keyword) Citation: Makin JG, Fellows MR and Sabes P (2011). Multisensory Integration via Density Estimation. Front. Comput. Neurosci. Conference Abstract: BC11 : Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meeting 2011. doi: 10.3389/conf.fncom.2011.53.00143 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 23 Aug 2011; Published Online: 04 Oct 2011. * Correspondence: Dr. Joseph G Makin, University of California at San Francisco, Department of Physiology and Keck Center for Integrative Neuroscience, San Francisco, United States, makin@phy.ucsf.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Joseph G Makin Matthew R Fellows Philip Sabes Google Joseph G Makin Matthew R Fellows Philip Sabes Google Scholar Joseph G Makin Matthew R Fellows Philip Sabes PubMed Joseph G Makin Matthew R Fellows Philip Sabes Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.