Neuromorphic sensorimotor adaptation for robotic mobile manipulation: From sensing to behaviour

Abstract

We propose a neuromorphic approach to perception, reasoning and motor control using Spiking Neural Networks in mobile robotics. We demonstrate this by using a mobile robotic manipulator solving a pick-and-place task. All sensory data is provided by spike-based silicon retina cameras - eDVS (embedded Dynamic Vision Sensor) - and all reasoning and motor control is implemented in Spiking Neural Networks. For the given scenario, the robot is capable of detecting a sequence of objects blinking at different frequencies, finding one object that is not in the right place of the sequence, picking up this object and moving it to its correct position. Such a scenario demonstrates how to build large-scale networks solving a high-level cognitive task by combining several smaller networks responsible for low-level tasks. Importantly, here we focus only on generating a neural network that is capable of performing the task. This will be the basis of future work using neural network learning algorithms to improve task performance. The long-term goal is to learn sophisticated behaviours by experience while at the same time being able to introduce expert knowledge for intermediate tasks that can be used to initialize the network or to speed up the learning process.