Adaptive Reinforcement Learning Research Articles

An automated FX trading system using adaptive reinforcement learning

This paper introduces adaptive reinforcement learning (ARL) as the basis for a fully automated trading system application. The system is designed to trade foreign exchange (FX) markets and relies on a layered structure consisting of a machine learning algorithm, a risk management overlay and a dynamic utility optimization layer. An existing machine-learning method called recurrent reinforcement learning (RRL) was chosen as the underlying algorithm for ARL. One of the strengths of our approach is that the dynamic optimization layer makes a fixed choice of model tuning parameters unnecessary. It also allows for a risk-return trade-off to be made by the user within the system. The trading system is able to make consistent gains out-of-sample while avoiding large draw-downs.

Behavior-based intelligent mobile robot using an immunized reinforcement adaptive learning mechanism

In this paper, a novel immunized reinforcement adaptive learning mechanism employing a behavior-based knowledge and the on-line adapting capabilities of the immune system is proposed and applied to an intelligent mobile robot. Rather than building a detailed mathematical model of immune systems, we try to explore principles in the immune system focusing on its self-organization, adaptive capability and immune memory. Two levels of the immune system, underlying the ‘micro’ level of cell interactions, and emergent ‘macro’ level of the behavior of the system are investigated. To evaluate the proposed immunized architecture, a ‘food foraging work’ simulation environment containing a mobile robot, foods, with/without obstacles is created to simulate the real world. The simulation results validate several significant characteristics of the immunized architecture: adaptability, learning, self-organizing, and stable ecological niche approaching.

FLNN에 기초한 XY Table용 마찰 보상 제어기

An FLNN-based neural network controller is applied to precise positioning of XY table with friction as the extension study of [11]. The neural network identifies the frictional farces of the table. Its weight adaptation rule, named the reinforcement adaptive learning rule, is derived from the Lyapunov stability theory. The experimental results with 2-DOF XY table verify the effectiveness of the proposed control scheme. It is also expected that the proposed control approach is applicable to a wide class of mechanical systems.

Reinforcement Adaptive Fuzzy Control for a Class of Nonlinear Uncertain Systems

The paper describes the application of reinforcement learning techniques to feedback control for a class of nonlinear systems using adaptive fuzzy system. The adaptive fuzzy system approximates the non linear characteristics of the plant on-line to achieve the desired tracking accuracy without any preliminary off-line learning or training phase. The reinforcement signal from a critic is used for finding proper fuzzy rules to approximate the non linear dynamics in the plant. The approximation of the nonlinear dynamics in the plant is performed on-line without knowing any knowledge of the nonlinear terms in the plant, whence comes the term Reinforcement Adaptive Learning (RAL). The fuzzy system based on the RAL algorithm is referred to as Reinforcement Adaptive Fuzzy (RAF) system. The RAL algorithm is derived from Lyapunov stability analysis, so that both tracking stability and error convergence can be guaranteed in the closed-loop system.

Reinforcement adaptive learning neural-net-based friction compensation control for high speed and precision

There is an increasing number of applications in high-precision motion control systems in manufacturing, i.e., ultra-precision machining, assembly of small components and micro devices. It is very difficult to assure such accuracy due to many factors affecting the precision of motion, such as frictions and disturbances in the drive system. The standard proportional-integral-derivative (PID) type servo control algorithms are not capable of delivering the desired precision under the influence of frictions and disturbances. In this paper, the frictions are identified by a neural net, which has a critic element to measure the system performance. Then, the weight adaptation rule, defined as reinforcement adaptive learning, is derived from the Lyapunov stability theory. Therefore the proposed scheme can be applicable to a wide class of mechanical systems. The simulation results on a 1-degree-of-freedom mechanical system verify the effectiveness of the proposed algorithm.

Adaptive reinforcement learning system for linearization control

A linearization scheme is proposed to demonstrate how a neural network scheme learns to linearize a system without any identification. The process occurs within an evaluator and a controller, which communicate with each other through reinforcement signals. From simulation results, the proposed learning scheme notably surpasses the conventional neural network approaches.

Introduction to the Special Issue on Learning in Intelligent Algorithms and Systems Design

Introduction to the Special Issue on Learning in Intelligent Algorithms and Systems Design

Adaptive Reinforcement Learning Integrating Exploitation-and Exploration-oriented Learning

We propose and evaluate an adaptive reinforcement learning system that integrates both exploitation- and exploration-oriented learning (ArLee). Compared to conventional reinforcement learning, ArLee is more robust in a dynamically changing environment and conducts exploration-oriented learning efficiently even in a large-scale environment. It is thus well suited for autonomous systems, for example, software agents and mobile robots, that operate in dynamic, large-scale environments, such as the real world and the Internet. Simulation demonstrates the learning system’s basic effectiveness.

Adaptive Reinforcement Learning and Its Application to Robot Compliance Learning

A new learning algorithm for connectionist networks that solves a class of optimal control problems is presented. The algorithm, called Adaptive Reinforcement Learning Algorithm, employs a second network to model immediate reinforcement provided from the task environment and adaptively identities it through repeated experience. Output perturbation and correlation techniques are used to translate mere critic signals into useful learning signals for the connectionist controller. Compared with the direct approaches of reinforcement learning, this algorithm shows faster and guaranteed improvement in the control performance. Robustness against inaccuracy of the model is also discussed. It is demonstrated by simulation that the adaptive reinforcement learning method is efficient and useful in learning a compliance control law in a class of robotic assembly tasks. A simple box palletizing task is used as an example, where a robot is required to move a rectangular part to the corner of a box. In the simulation, the robot is initially provided with only predetermined velocity command to follow the nominal trajectory. At each attempt, the box is randomly located and the part is randomly oriented within the grasp of the end-effector. Therefore, compliant motion control is necessary to guide the part to the corner of the box while avoiding excessive reaction forces caused by the collision with a wall. After repeating the failure in performing the task, the robot can successfully learn force feedback gains to modify its nominal motion. Our results show that the new learning method can be used to learn a compliance control law effectively.