Learning strategy for continuous robot visual control: A multi-objective perspective

Abstract

Robot visual control aims to achieve three general objectives, namely, smoothness, rapidity, and target keeping. In practice, such conflicting objectives make robot visual control, which is often formulated as a multi-objective optimization problem (MOP), difficult to achieve. Conventional solutions for MOP set constant weights to the objectives throughout the decision process. However, in practice, a robot focuses on different objectives in different motion phases. Thus, time-varying visual control is desired. Deep Reinforcement Learning (DRL) is a promising solution to handle time-varying decisions in the MOP domain. Renowned DRL solutions suffer from high computing costs and low data efficiency when handling real-time visual control. To satisfy the control requirements and improve the learning efficiency when applying DRL to robot visual control, a lightweight DRL solution, referred to as Fuzzy Cerebellar Actor–critic (FCAC), is developed in this paper. In FCAC, Fuzzy Coding is employed to represent continuous observations. The policy is evaluated by a set of embedding vectors consisting of weighted states. Then, based on the observation error, a stochastic Actor–critic policy is learned to compute a suitable continuous control gain. To evaluate the performance of the proposed FCAC in robust control, we have simulated different general robot tasks. Experimental results show that robots perform very well by the DRL-driven strategies with diverse controllers under noise interference. Meanwhile, the FCAC shows higher learning efficiency and lower cost compared to the existing DRL solutions in the MOP domain.