A Novel Hierarchical Soft Actor-Critic Algorithm for Multi-Logistics Robots Task Allocation

Hengliang Tang,Anqi Wang,Jiaxin Yang,Yang Cao,Fei Xue

doi:10.1109/access.2021.3062457

Abstract

In intelligent unmanned warehouse goods-to-man systems, the allocation of tasks has an important influence on the efficiency because of the dynamic performance of AGV robots and orders. The paper presents a hierarchical Soft Actor-Critic algorithm to solve the dynamic scheduling problem of orders picking. The method proposed is based on the classic Soft Actor-Critic and hierarchical reinforcement learning algorithm. In this paper, the model is trained at different time scales by introducing sub-goals, with the top-level learning a policy and the bottom level learning a policy to achieve the sub-goals. The actor of the controller aims to maximize expected intrinsic reward while also maximizing entropy. That is, to succeed at the sub-goals while moving as randomly as possible. Finally, experimental results for simulation experiments in different scenes show that the method can make multi-logistics AGV robots work together and improves the reward in sparse environments about 2.61 times compared to the SAC algorithm.

Highlights

The logistics industry has entered the era of intelligent logistics [1] which is highly informative, automated, intelligent and networked
We present the Hierarchical Soft Actor-Critic algorithm(HSAC), which builds on the Soft Actor-Critic algorithm by applying the modifications describes in the last section to solve dimensional disaster problem with consideration of hierarchical architecture
The proposed HSAC algorithm takes robot environment observations as input of neural network to directly control AGV robots to shuttle between shelves and picking stations in a dynamic environment

Summary

Introduction

The logistics industry has entered the era of intelligent logistics [1] which is highly informative, automated, intelligent and networked. The efficient operation of each part of the intelligent logistics system requires the support of the intelligent storage system. The intelligent storage system [2], [3] uses the Internet of things technology to sense the storage status in real-time, and applies artificial intelligence technology for data processing and analysis. Compared with the traditional storage system, the intelligent storage system has higher efficiency, higher fault tolerance rate, lower labor cost, and strong robustness. A large amount of information will be generated during the operation of the intelligent storage system, which is characterized by the dynamic nature of order information, goods information and storage information. A large number of warehousing logistics robots and artificial intelligence technologies are needed to

Objectives

Methods

Conclusion