Rapid Reinforcement Learning Research Articles

Rapid Reinforcement Learning by Injecting Stochasticity into Bellman

The Courchevel environment is hereby published to ease the development of streaming machine learning algorithms. In first solving this problem, a rapid reinforcement learning algorithm was invented. A simple transformation is added to the Bellman equation, a principal pillar of AI, particularly for solving Markov Decision Problems. By adding stochasticity to Bellman, sustained Reward-Per-Episode gains of an order of magnitude are validated, for environments where the reward function is structurally anticipated to be multi-modal. Courchevel as a decision problem, a first solution, and the Biased Bellman innovation are revealed -- with accompanying data. For ease of discussion, Courchevel's dynamics are described in military terms.

The Roles of Reward, Default, and Executive Control Networks in Set-Shifting Impairments in Schizophrenia

Patients with schizophrenia (SZ) show deficits on tasks of rapid reinforcement learning, like probabilistic reversal learning (PRL), but the neural bases for those impairments are not known. Recent evidence of relatively intact sensitivity to negative outcomes in the ventral striatum (VS) in many SZ patients suggests that PRL deficits may be largely attributable to processes downstream from feedback processing, involving both the activation of executive control task regions and deactivation of default mode network (DMN) components. We analyzed data from 29 chronic SZ patients and 21 matched normal controls (NCs) performing a PRL task in an MRI scanner. Subjects were presented with eight pairs of fractal stimuli, for 50 trials each. For each pair, subjects learned to choose the more frequently-rewarded (better) stimulus. Each time a criterion was reached, the better stimulus became the worse one, and the worse became the better. Responses to feedback events were assessed through whole-brain and regions-of-interest (ROI) analyses in DMN. We also assessed correlations between BOLD signal contrasts and clinical measures in SZs. Relative to NCs, SZ patients showed comparable deactivation of VS in response to negative feedback, but reduced deactivation of DMN components including medial prefrontal cortex (mPFC). The magnitudes of patients' punishment-evoked deactivations in VS and ventromedial PFC correlated significantly with clinical ratings for avolition/anhedonia. These findings suggest that schizophrenia is associated with a reduced ability to deactivate components of default mode networks, following the presentation of informative feedback and that motivational deficits in SZ relate closely to feedback-evoked activity in reward circuit components. These results also confirm a role for ventrolateral and dorsomedial PFC in the execution of response-set shifts.

Rapid Reinforcement Learning for Reactive Control Policy Design in Autonomous Robots

Rapid Reinforcement Learning for Reactive Control Policy Design in Autonomous Robots

058 Rapid reinforcement learning methods for robots with discrete state representation

058 Rapid reinforcement learning methods for robots with discrete state representation

Rapid Reinforcement Learning Methods for Robots with Discrete State Representations

This paper describes an autonomous mobile robot which learns to find its way through an arbitrary maze environment to a location at which it receives positive reinforccment, while avoiding locations at which it receives negative reinforccment. The robot represents its current state in terms of one of a large numbcr of possible discrete states defined by the current sensory input and the memory of the last few sensory inputs and output actions. Thc basic learning algorithm is Q-learning; enhancements are described which can spced up learning by generalising across states (generalisation-Q), and by building and exploiting a deictic world model (implication-Q).

Reinforcement Learning and Recruitment Mechanism for Adaptive Distributed Control

The work presented in this paper is an attempt to spread further the inspiration gained from the knowledge of biological systems into the field of adaptive control. After the neural controllers and the evolutionary based mechanisms, new hints for the control of complex processes might be derived from other biological domains such as immunology or the study of conditioning learning. The conception of a system equipped with a complex controller, interacting with an uncertain and varying environment, and basing its learning on its own experiences entails quite naturally the integration of a reinforcement learning mechanism. Two learning processes characterized by two different time scales will be introduced, will be connected to their respective biological origins and will be illustrated on the classical cart-pole control problem. These two learning processes are the rapid reinforcement learning and the slower recruitment mechanism.

Reinforcement learning and recruitment mechanism for adaptive distributed control

The work presented in this paper is an attempt to spread further the inspiration gained from the knowledge of biological systems into the field of adaptive control. After the neural controllers and the evolutionary based mechanisms, new hints for the control of complex processes might be derived from other biological domains such as immunology or the study of conditioning learning. The conception of a system equipped with a complex controller, interacting with an uncertain and varying environment, and basing its learning on its own experiences entails quite naturally the integration of a reinforcement learning mechanism. Two learning processes characterized by two different time scales will be introduced, will be connected to their respective biological origins and will be illustrated on the classical cart-pole control problem. These two learning processes are the rapid reinforcement learning and the slower recruitment mechanism.