An autonomous ore packing system through deep reinforcement learning

Abstract

In the contemporary era, the limited availability of terrestrial resources has prompted an increasing number of nations to turn their attention towards space, wherein extraterrestrial minerals hold considerable allure for both resource provisioning and scientific inquiry. Numerous nations have initiated the deployment of unmanned operational platforms towards extraterrestrial asteroids with the objective of accomplishing sampling return missions. Given the restricted storage capacity and energy limitations of these platforms, the optimization of mineral placement algorithms assumes paramount importance in enhancing the efficacy of these missions. In this paper, we propose an autonomous ore packing system capable of autonomously measuring ore characteristics and addressing the ore packing optimization problem in extraterrestrial unfamiliar environments. A deep reinforcement learning method that utilizes physical constraints to enhance overall performance was proposed to solve the ore packing problem with uncertainty, while meeting the high demands for real-time performance in the mission. To augment the autonomy and adaptability of our approach, we leverage advanced visual technology to transform the spatial distribution of bin utilization and the characteristics of the ore into a matrix representation. This empowers the robotic system to autonomously perceive bin information and capture essential ore features. The empirical findings substantiate that our algorithm attains a human-level performance in the majority of instances, rendering an approximate optimal solution within a concise timeframe. Additionally, we introduce a novel reinforcement learning training technique known as Maximum Worth Reinforcement Learning (MWRL) to address the optimization conundrum associated with incomplete Markov chains, which outperforms existing approaches in our comparative analysis. Lastly, we validate the efficacy of our algorithm in real-world scenarios by deploying it on a robotic manipulator.