A dual-memory architecture for reinforcement learning on neuromorphic platforms

Wilkie Olin-Ammentorp,Maxim Bazhenov,Yury Sokolov

doi:10.1088/2634-4386/ac1a64

Abstract

Reinforcement learning (RL) is a foundation of learning in biological systems and provides a framework to address numerous challenges with real-world artificial intelligence applications. Efficient implementations of RL techniques could allow for agents deployed in edge-use cases to gain novel abilities, such as improved navigation, understanding complex situations and critical decision making. Toward this goal, we describe a flexible architecture to carry out RL on neuromorphic platforms. This architecture was implemented using an Intel neuromorphic processor and demonstrated solving a variety of tasks using spiking dynamics. Our study proposes a usable solution for real-world RL applications and demonstrates applicability of the neuromorphic platforms for RL problems.

Highlights

As the number of data-collecting devices increases, so too does the need for efficient data processing
Dual-memory learner (DML) framework Monte Carlo (MC) methods provide well-characterized Reinforcement learning (RL) techniques for learning optimal policies via episodic experiences; the agent does not need to be equipped with a full model of how the environment will react to its actions in order to learn
It is crucial for neuromorphic systems to show that they are capable of RL techniques and can demonstrate advantages for these techniques over traditional hardware

Summary

Introduction

As the number of data-collecting devices increases, so too does the need for efficient data processing. Rather than require all data collected from remote devices be processed at a central location, the need for data processing to be performed in-situ is becoming a priority; this is especially true in situations where ‘agents’ collecting data may need to make critical decisions based on these inputs with low latency (such as in self-driving cars or aerial drones) For such use cases, efficiency of data processing becomes paramount, as energy sources and physical space (‘size, weight, and power’) come at a premium[1]. There is no universal definition on what constitutes a neuromorphic architecture, these systems generally aim to provide efficient, massively-parallel processing schemes which often use binary ‘spikes’ to transmit information[2]. We utilize Intel’s neuromorphic processor codenamed ‘Loihi.’[4]

Methods

Results

Discussion

Conclusion