Solving biobjective traveling thief problems with multiobjective reinforcement learning

Abstract

This study proposes an end-to-end multiobjective reinforcement learning (MORL) approach to solve the biobjective traveling thief problems (TTP). A TTP involves a thief visiting cities and selecting items to maximize profit while minimizing travel time within a knapsack’s capacity. The study evaluates combinations of two architectures, namely the pointer network (PN) and attention mechanism (AM), with three MORL methods: deep reinforcement learning multiobjective algorithm (DRLMOA), multi-sample Pareto hypernetwork (PHN), and manifold-based policy search (MBPS). However, PN and AM cannot be directly used to predict two different sequences simultaneously: the city tour and the item selection. Therefore, a solution encoding and decoding scheme is proposed to solve TTP without substantially modifying PN and AM. The methods are trained on only small randomly generated problem instances based on Eil76 instances, and their performance is evaluated on various problem instances. The state-of-the-art non-dominated sorting-based customized random-key genetic algorithm (NDS-BRKGA) serves as the baseline. The experimental study demonstrates a competitive performance of the proposed methods compared to the baseline, particularly in instances with a high number of items. The proposed methods, especially PN-DRLMOA and AM-DRLMOA, also show promising generalization capabilities on different and larger graphs. Lastly, the proposed MORL methods significantly outperform NDS-BRKGA in terms of the solution generation running time.