Generation Of Control Policies Research Articles

Transferring human emotions to robot motions using Neural Policy Style Transfer

Neural Style Transfer (NST) was originally proposed to use feature extraction capabilities of Neural Networks as a way to perform Style Transfer with images. Pre-trained image classification architectures were selected for feature extraction, leading to new images showing the same content as the original but with a different style. In robotics, Style Transfer can be employed to transfer human motion styles to robot motions. The challenge lies in the lack of pre-trained classification architectures for robot motions that could be used for feature extraction. Neural Policy Style Transfer TD3 (NPST3) is proposed for the transfer of human motion styles to robot motions. This framework allows the same robot motion to be executed in different human-centered motion styles, such as in an “angry”, “happy”, “calm”, or “sad” fashion. The Twin Delayed Deep Deterministic Policy Gradient (TD3) network is introduced for the generation of control policies. An autoencoder network is in charge of feature extraction for the Style Transfer step. The Style Transfer step can be performed both offline and online: offline for the autonomous executions of human-style robot motions, and online for adapting at runtime the style of e.g., a teleoperated robot. The framework is tested using two different robotic platforms: a robotic manipulator designed for telemanipulation tasks, and a humanoid robot designed for social interaction. The proposed approach was evaluated for both platforms, performing a total of 147 questionnaires asking human subjects to recognize the human motion style transferred to the robot motion for a predefined set of actions.

Attribute-Based Access Control Policy Generation Approach from Access Logs Based on the CatBoost

Attribute-Based Access Control Policy Generation Approach from Access Logs Based on the CatBoost

Event-Triggered Model Predictive Adaptive Dynamic Programming for Road Intersection Path Planning of Unmanned Ground Vehicle

Autonomous driving of unmanned ground vehicle (UGV) at road intersection is a challenging task due to the complicated traffic conditions. In this paper, an event-triggered model predictive adaptive dynamic programming (MPADP) algorithm is proposed for path planning of UGV at road intersection. Following the critic-actor scheme of adaptive dynamic programming (ADP), cost function approximation and control policy generation are combined to formulate MPADP. The infinite horizon cost function of ADP is stacked over predictive horizon of model predictive control (MPC), and then the infinite horizon cost function is converted to the finite horizon-stacked cost function in MPADP. By minimizing the approximation error within predictive horizon, the approximation accuracy is enhanced. Considering the limitation of energy consumption, the event-triggered mechanism is designed based on the mismatch of cost function approximation. Three triggering conditions are designed, and the corresponding boundedness of approximation error is proved. Simulation results illustrate the effectiveness, efficiency and feasibility in application of the event-triggered MPADP method for path planning at road intersection.

Undiscounted control policy generation for continuous-valued optimal control by approximate dynamic programming

We present a numerical method for generating the state-feedback control policy associated with general undiscounted, constant-setpoint, infinite-horizon, nonlinear optimal control problems with continuous state variables. The method is based on approximate dynamic programming, and is closely related to approximate policy iteration. Existing methods typically terminate based on the convergence of the control policy and either require a discounted problem formulation or demand the cost function to lie in a specific subclass of functions. The presented method extends on existing termination criteria by requiring both the control policy and the resulting system state to converge, allowing for use with undiscounted cost functions that are bounded and continuous. This paper defines the numerical method, derives the relevant underlying mathematical properties, and validates the numerical method with representative examples. A MATLAB implementation with the shown examples is freely available.

Interpretable policy derivation for reinforcement learning based on evolutionary feature synthesis

Reinforcement learning based on the deep neural network has attracted much attention and has been widely used in real-world applications. However, the black-box property limits its usage from applying in high-stake areas, such as manufacture and healthcare. To deal with this problem, some researchers resort to the interpretable control policy generation algorithm. The basic idea is to use an interpretable model, such as tree-based genetic programming, to extract policy from other black box modes, such as neural networks. Following this idea, in this paper, we try yet another form of the genetic programming technique, evolutionary feature synthesis, to extract control policy from the neural network. We also propose an evolutionary method to optimize the operator set of the control policy for each specific problem automatically. Moreover, a policy simplification strategy is also introduced. We conduct experiments on four reinforcement learning environments. The experiment results reveal that evolutionary feature synthesis can achieve better performance than tree-based genetic programming to extract policy from the neural network with comparable interpretability.

A learning machine with monologue

The learning machine STeLLA has been developed considerably since it was first described in 1962. Although it has not been built in entirety, it has been simulated on computers in many different forms and with many different problems; special circuits have been developed for its construction. The need to give the machine a “monologue” ability arose because the machine cannot learn to solve problems for which the input to the machine is inadequate to distinguish successive steps made by the machine. A simple example of this is the problem of learning to perform a sequence of actions when the sequence is not related to information received by the machine from its problem environment. In this paper an introductory description of the STeLLA machine is given with the help of a particular problem which is then used to illustrate the generation of control policies by a dual machine. The dual STeLLA comprises two interdependent STeLLA machines or STeLLAments; one interacts directly with the problem environment, while the second STeLLAment interacts with an auxiliary “vocal” environment to provide monologue. Monologue is used to supplement the information from the problem invironment with information from the vocal environment. The STeLLAment interacting directly with the vocal environment has its input supplemented by information from the control policy of the other machine. The two machines are co-ordinated further by giving reward to the dual machine as a whole. The procedure of counting is the kind of monologue which can be used to distinguish a sequence of steps that cannot be distinguished by successive inputs from the problem environment. This is illustrated in the paper by considering a problem that requires STeLLA “to walk across a dark courtyard”. Monologue is not restricted to such counting of steps, but can take the form of more sophisticated “symbol” sequences. By allowing an operator to inject sequences into the monologue of the machine, an elementary form of dialogue could be set up. It would be a special feature of the dialogue that no predetermined language was imposed on the participants. Examples of the use of monologue have been worked out and more complicated situations are being programmed for computer simulation both in the monologue and dialogue form.