Abstract
In embodied computation (or morphological computation), part of the complexity of motor control is offloaded to the body dynamics. We demonstrate that a simple Hebbian-like learning rule can be used to train systems with (partial) embodiment, and can be extended outside of the scope of traditional neural networks. To this end, we apply the learning rule to optimize the connection weights of recurrent neural networks with different topologies and for various tasks. We then apply this learning rule to a simulated compliant tensegrity robot by optimizing static feedback controllers that directly exploit the dynamics of the robot body. This leads to partially embodied controllers, i.e., hybrid controllers that naturally integrate the computations that are performed by the robot body into a neural network architecture. Our results demonstrate the universal applicability of reward-modulated Hebbian learning. Furthermore, they demonstrate the robustness of systems trained with the learning rule. This study strengthens our belief that compliant robots should or can be seen as computational units, instead of dumb hardware that needs a complex controller. This link between compliant robotics and neural networks is also the main reason for our search for simple universal learning rules for both neural networks and robotics.
Highlights
Hebbian theory has been around for over half a century (Hebb, 1949), but it still sparks the interest of today’s researchers
In previous work (Caluwaerts et al, 2012), we demonstrated that the motor control of a tensegrity robot can be drastically simplified by using its body as a computational resource
We show that the reward modulated Hebbian (RMH) learning rule can be extended to systems exhibiting partial embodiment, i.e., agents that actively see and use their body as a computational resource
Summary
Hebbian theory has been around for over half a century (Hebb, 1949), but it still sparks the interest of today’s researchers. The basic rule is biologically plausible as are some of its variations (Mazzoni et al, 1991; Loewenstein and Seung, 2006) Whereas all these approaches belong to the general category of unsupervised learning, reward modulated Hebbian (RMH) learning is similar to reinforcement learning in that it can be used to tune a neural system to solve a specific task without the need to know the desired output signals at the neural level (Fiete and Seung, 2006; Legenstein et al, 2010; Hoerzer et al, 2012; Soltoggio and Steil, 2013; Soltoggio et al, 2013). There is no need for complex mathematical operations and it can be efficiently implemented on various platforms in hardware and software It allows for a distributed implementation: a central unit can be responsible for a global reward, which can be broadcast to the learning units of local controllers. If the reward mechanism remains active, the controller can adapt to changes in the robot morphology or dynamics, e.g., due to wear or damage
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
