Solving the binary knapsack problem using tabular and deep reinforcement learning algorithms

Abstract

The knapsack problem (KP) is a traditional optimization problem. Its applicability in the real-world ranges from route selection in logistics and portfolio optimization in finance to energy and server load balancing. Within this study, we tested the viability and effectiveness of the use of Reinforcement Learning (RL) algorithms to solve the KP. Multiple experiments conducted to test performance of different Tabular RL and Deep RL methods. These results are compared with classic optimization techniques such as dynamic programming (DP) and metaheuristic approaches. Results show that RL methods are capable of adapting to changes in the size of the KP with minor tuning to hyperparameters. Within the RL category, the best overall performer was the Double Deep Q-Network (DDQN) algorithm which utilizes two neural networks to find the best policy. This algorithm was able to achieve the highest total value in the knapsack for the most experiment scenarios (including metaheuristic algorithms). This project sets a steppingstone for analyzing the applicability of RL in traditional optimization problems such as KP. The results of this study can provide insights for future works and enable researchers to extend the work to other RL algorithms and applications including stochastic environments.--Author's abstract