Approximative Pareto Front Identification

Abstract

Techniques from multi-objective optimization are incorporated into the stochastic multi-armed bandit (MAB) problem to improve performance when the rewards obtained from pulling an arm are random vectors instead of random variables. We call this problem the stochastic multi-objective MAB (or MOMAB) problem. In this paper, we study the analytical and empirical proprieties of MOMABs with the goal of identifying multiple arms in the Pareto front that use the partial Pareto dominance relation to compare mean reward vectors. We introduce three algorithms: 1) Pareto Front Identification identifies the Pareto optimal arms using a fixed budget. 2) e-approximate Pareto Front Identification uses the Pareto e-dominance to identify a uniformly spread subset of the Pareto front. 3) Pareto Sub front Identification combines the last two algorithms to improve the accuracy of the e-approximation Pareto front. We experimentally compare the proposed algorithms on several Pareto MAB-problems.