A spiking neural network model for obstacle avoidance in simulated prosthetic vision

Abstract

Limited by visual percepts elicited by existing visual prosthesis, it’s necessary to enhance its functionality to fulfill some challenging tasks for the blind such as obstacle avoidance. This paper argues that spiking neural networks (SNN) are effective techniques for object recognition and introduces for the first time a SNN model for obstacle recognition to assist blind people wearing prosthetic vision devices by modelling and classifying spatio-temporal (ST) video data. The proposed methodology is based on a novel spiking neural network architecture, called NeuCube as a general framework for video data modelling in simulated prosthetic vision. As an integrated environment including spiking trains encoding, input variable mapping, unsupervised reservoir training and supervised classifier training, the NeuCube consists of a spiking neural network reservoir (SNNr) and a dynamic evolving spiking neural network classifier (deSNN). First, input data is captured by visual prosthesis, then ST feature extraction is utilized in the low-resolution prosthetic vision generated by prostheses. Finally such ST features are fed to the NeuCube to output classification result of obstacle analysis for an early warning system to be activated. Experiments on collected video data and comparison with other computational intelligence methods indicate promising results. This makes it possible to directly utilize available neuromorphic hardware chips, embedded in visual prostheses, to enhance significantly their functionality. The proposed NeuCube-based obstacle avoidance methodology provides useful guidance to the blind, thus offering a significant improvement of current prostheses and potentially benefiting future prosthesis wearers.