Towards learning behavior modeling of military logistics agent utilizing profit sharing reinforcement learning algorithm

Abstract

Agent-based modeling has become a beneficial tool in describing the complex and intelligent decision-making behaviors of military logistics entities, which is essential in exploring military logistics system. A challenging task in this field is the learning behavior modeling of military logistics agents. Profit sharing (PS) reinforcement learning algorithm is a representative exploitation-oriented method describing empirical reinforcement learning mechanism, and has been successfully applied to a variety of real-world problems. However, constructing the learning behavior model of military logistics agents is difficult by merely using the original PS algorithm. This difficulty is due to the actual characteristics of equipment support operations and military requirements, such as experience sharing, cooperative action, and hierarchical control. To address this issue, we propose an improved PS algorithm by introducing cooperative task reward correction parameters, experience sharing learning function, and superior command controlled function. We use the research methodology centering on the basic process of the improved PS algorithm as basis to construct the architecture of the learning behavior model of military logistics agents and its corresponding model of elements. Furthermore, we design the implementation algorithm of the learning behavior model. Lastly, we conduct a case study of a tactical military industrial logistics simulation system, thereby verifying the feasibility and effectiveness of the learning behavior model. We find that the improved PS algorithm and corresponding learning behavior model have more advantages than the original PS algorithm.