Learning manipulation skills with demonstrations for the swing process control of dredgers

Abstract

The Cutter Suction Dredger (CSD) is one of the key equipment dedicated to the construction and maintenance projects of harbours, ports and navigational channels. Among the dredging manipulations, the swing process is the most tedious and recurring work for human operators, which often leads to accidents because of carelessness or fatigue of the operators. This paper aims at producing a learning approach for the intelligent control of the swing process of a CSD so as to release human operators from such a boring and heavy task. To this end, the swing process control is formulated as a sequential decision making problem, and Deep Reinforcement Learning (DRL) is employed to design the learning approach based on deterministic policy gradient. The novel feature of the proposed approach is that the manipulation skills are obtained via trial-and-error interactions with a predicting network constructed by human demonstration data. In our approach, human demonstrations can provide a channel to predict state transitions, and also can regulate the exploration procedure for the learning agent. In addition, we carry out empirical studies to investigate how to treat the demonstration data with regard to self exploration, and the experimental results show that the proposed approach provides an effective means of controlling the swing process of CSDs.