예산제약이 존재하는 다기간, 다제품 재고관리문제에서 강화학습 기법의 효율성 개선 방안 연구

Abstract

This paper considers the use of reinforcement learning for a multi-period, multi-item inventory control problem with a budget constraint. In the problem, we decide the order quantities of multiple items considering budget constraints so as to minimizes the total inventory cost including inventory holding cost and backlog cost. The previous literature proposed a modified Q-learning that include an optimization model in the Q-learning procedure to handle budget constrained actions, but it lacks the scalability. To address this issue, this paper proposed a two-stage method: the Q-learning learns actions without considering the budget constraint in the first stage, and an optimization model adjusts the learned actions so as to satisfy the budget constraint in the second stage. Numerical study compares the performance of the proposed two-stage method with others such as a conventional Q-learning without the budget constraint and the modified Q-learning in the literature. The numerical experiments reveal that the proposed method significantly reduces the computation time without increasing the total inventory cost.