Increasing the Energy-Efficiency in Vacuum-Based Package Handling Using Deep Q-Learning

Felix Gabriel,Klaus Dröder,Johannes Bergers,Franziska Aschersleben

doi:10.3390/en14113185

Abstract

Billions of packages are automatically handled in warehouses every year. The gripping systems are, however, most often oversized in order to cover a large range of different carton types, package masses, and robot motions. In addition, a targeted optimization of the process parameters with the aim of reducing the oversizing requires prior knowledge, personnel resources, and experience. This paper investigates whether the energy-efficiency in vacuum-based package handling can be increased without the need for prior knowledge of optimal process parameters. The core method comprises the variation of the input pressure for the vacuum ejector, compliant to the robot trajectory and the resulting inertial forces at the gripper-object-interface. The control mechanism is trained by applying reinforcement learning with a deep Q-agent. In the proposed use case, the energy-efficiency can be increased by up to 70% within a few hours of learning. It is also demonstrated that the generalization capability with regard to multiple different robot trajectories is achievable. In the future, the industrial applicability can be enhanced by deployment of the deep Q-agent in a decentral system, to collect data from different pick and place processes and enable a generalizable and scalable solution for energy-efficient vacuum-based handling in warehouse automation.

Highlights

Vacuum-based handling is used in a large variety of applications, especially when a high flexibility is required due to diverse objects that must be grasped, e.g., in packaging and warehouse logistics
Current vacuum-based gripping systems for package handling are mostly realized by means of compressed air-supplied vacuum ejectors and exhibit a highly dynamic and wear-free operation
Using vacuum ejectors causes enormous energy losses [2,3], since only a few percent of the initially provided electrical energy can be utilized for the handling process

Summary

Introduction

Vacuum-based handling is used in a large variety of applications, especially when a high flexibility is required due to diverse objects that must be grasped, e.g., in packaging and warehouse logistics. These fields of application are constantly gaining relevance, as global retail e-commerce sales amounted to 4.3 trillion US dollars in 2020 and revenues are estimated to grow to 6.4 trillion dollars until 2024 [1]. It is crucial to design the gripping system and the corresponding process parameters in compliance with the application-specific requirements such as the expected robot trajectories and the properties of the objects to be handled. In case of a large spectrum of objects to be handled, it is usually not economically feasible to adjust the process parameters for each specific trajectory and object

Objectives

Methods

Results

Conclusion