Form Of Reinforcement Learning Research Articles

The human ventromedial frontal lobe is critical for learning from negative feedback

Are positive and negative feedback weighed in a common balance in the brain, or do they influence behaviour through distinct neural mechanisms? Recent neuroeconomic studies in both human and non-human primates indicate that the ventromedial frontal lobe carries information about both losses and gains, suggesting that this region may encode value across the continuum from absolute negative to absolute positive outcomes. However, such work does not specify whether or how this value information is applied during behaviour. Observations of patients with ventromedial frontal damage indicate that this region is critical for certain forms of reinforcement learning and value-based decision-making, but the underlying processes remain unclear. We disentangled the influence of cumulative positive and negative feedback on subsequent behaviour with a probabilistic reinforcement learning task in 11 patients with ventromedial frontal damage, 9 lesioned controls and 24 healthy controls, and found that ventromedial frontal damage selectively disrupted the ability to learn from negative feedback.

Reinforcement Learning with Approximation Spaces

This paper introduces a rough set approach to reinforcement learning by swarms of cooperating agents. The problem considered in this paper is how to guide reinforcement learning based on knowledge of acceptable behavior patterns. This is made possible by considering behavior patterns of swarms in the context of approximation spaces. Rough set theory introduced by Zdzislaw Pawlak in the early 1980s provides a ground for deriving pattern-based rewards within approximation spaces. Both conventional and approximation space-based forms of reinforcement comparison and the actor-critic method as well as two forms of the off-policy Monte Carlo learning control method are investigated in this article. The study of swarm behavior by collections of biologically-inspired bots is carried out in the context of an artificial ecosystem testbed. This ecosystem has an ethological basis that makes it possible to observe and explain the behavior of biological organisms that carries over into the study of reinforcement learning by interacting robotic devices. The results of ecosystem experiments with six forms of reinforcement learning are given. The contribution of this article is the presentation of several viable alternatives to conventional reinforcement learning methods defined in the context of approximation spaces.

Strategy Acquisition for Games Based on Simplified Reinforcement Learning Using a Strategy Network

We propose a simplified form of reinforcement learning (RL) for game strategy acquisition using a strategy network. RL has been applied to a number of games, such as backgammon, checkers, etc. However, the application of RL to Othello or Shogi, which have very large state spaces, is more difficult because these games take a very long time to learning. The proposed strategy network is composed of N lines from N nodes on the game board with a single evaluation node as a 2-layer perceptron. These nodes denote all possible states of every square on the game board and can easily represent the evaluation function. Moreover, these nodes can also denote imaginary states, such as pieces that may exist in the next step, or denote every positional relation of two arbitrary pieces or other various board phases. After several thousands of games had been played, the strategy network quickly acquired a better evaluation function than that using a normalized Gaussian network. The computer player employing the strategy network beat a heuristic-based player that evaluates the values of pieces or places on the game board. The proposed strategy network was able to acquire good weightings of various features of game states. In addition, the player employing the strategy network for a 4×4 Othello task after co-evolutionary training acquired a winning strategy.