Bayesian Policy Optimization for Waste Crane With Garbage Inhomogeneity

Hikaru Sasaki,Terushi Hirabayashi,Kaoru Kawabata,Takamitsu Matsubara,Yukio Onuki

doi:10.1109/lra.2020.3002204

Hikaru Sasaki, Terushi Hirabayashi + Show 3 more

Open Access

https://doi.org/10.1109/lra.2020.3002204

Copy DOI

Abstract

The objective of this study is to develop a framework that can optimize control policies of a waste crane at a waste incineration plant through an autonomous trial and error manner. Since a waste crane is a massive mechanical system that moves slowly and takes several minutes to execute a task, obtaining data samples by executing tasks is very costly. Moreover, no sensors are available that can observe the state of the grasped flammable waste composed of various materials with different degrees of hardness and wetness. Therefore, the inhomogeneity of waste causes unpredictable fluctuation in the crane's task performance. To cope with these problems, we propose a framework for optimizing the policy parameters of a parameterized control policy with Multi-Task Robust Bayesian Optimization (MTRBO). Our framework features the following two characteristics: (1) outlier robustness against garbage inhomogeneity and (2) sample reuse from previously solved tasks to enhance its sample efficiency. To investigate the effectiveness of our framework, we conducted experiments on garbage-scattering tasks with (i) a robot waste crane with pseudo-garbage and (ii) an actual waste crane at a waste incineration plant. Experimental results demonstrate that our framework robustly optimized the control policies of the garbage cranes, even with a much reduced amount of data under the influence of garbage inhomogeneity.

Highlights

A UTOMATIC control of large industrial cranes is an attractive alternative to the use of human workers in high repetition and high-risk environments
The objective of this study is to develop a framework that can optimize control policies of a waste crane at a waste incineration plant through an autonomous trial and error manner based on a sample-efficient black-box optimization scheme so-called Bayesian optimization (BO) [19]–[21]
We evaluate the performance of Multi-Task Robust Bayesian Optimization (MTRBO) by comparing the evaluation value of the optimized parameters and the number of trials for the optimization with BO and MTRBO without reuse

Summary

Introduction

A UTOMATIC control of large industrial cranes is an attractive alternative to the use of human workers in high repetition and high-risk environments. Repetitive control of heavy-duty construction machinery is a heavy-labor task, so accidents at worksites remain high [1]. Integrating automatic control policies in such environments as waste incineration plants [2]–[4] is desirable. Automated crane and bucket control remains a challenging task, due to the difficulty in modeling their kinematic and dynamic characteristics, as often requires sophisticated nonlinear controllers specific to the system [5]. Date of publication June 15, 2020; date of current version June 22, 2020. This letter was recommended for publication by Associate Editor S. Choi upon evaluation of the reviewers’ comments. (Corresponding author: Hikaru Sasaki.)

Objectives

Methods

Results

Conclusion