Online learning for min-max discrete problems

Abstract

We study various discrete nonlinear combinatorial optimization problems in an online learning framework. In the first part, we address the computational complexity of designing vanishing regret (and vanishing approximate regret) algorithms. We provide a general reduction showing that many (min-max) polynomial time solvable problems not only do not have a vanishing regret, but also no vanishing approximation α-regret, for some α, unless NP=RP. In particular, for the min-max version of the vertex cover problem, which is solvable in polynomial time in the offline case, our reduction implies that there is no (2−ϵ)-regret online randomized algorithm unless Unique Game is in RP. Besides, we prove that the bound is tight by providing an online efficient algorithm based on the online gradient descent method. In the second part, we turn our attention to online learning algorithms that are based on an offline optimization oracle that, given a set of multiple instances of the problem, is able to compute the optimum static solution that performs best on the set of instances overall. We show that for several min-max (nonlinear) discrete optimization problems, it is strongly NP-hard to solve the offline optimization oracle, even for problems that can be solved in polynomial time in the single-instance static case (e.g. min-max vertex cover, min-max perfect matching, etc.). This also provides a useful insight into the connection between the non-linear nature of some problems and the drastic change of their computational hardness when moved to an online learning setting.