Safe Linear Thompson Sampling With Side Information

Abstract

The design and performance analysis of bandit algorithms in the presence of stage-wise safety or reliability constraints has recently garnered significant interest. In this work, we consider the linear stochastic bandit problem under additional unknown linear safety constraints that need to be satisfied at each round. For this problem, we present and analyze a new safe algorithm based on linear Thompson Sampling (TS). Our analysis shows that, with high probability, the algorithm chooses actions that are safe at each round and achieve cumulative regret of order <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (d <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3/2</sup> log <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sup> d ·T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</sup> log <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3/2</sup> T). Remarkably, this matches the regret bound provided by [1], [2] for the standard linear TS algorithm in the absence of safety constraints. Also, our analysis highlights how the inherently randomized nature of the TS selection rule suffices to properly expand the set of safe actions that the algorithm has access to at each round. In particular, we compare this behavior to alternative safe algorithms, which typically require distinct rounds of randomization that are dedicated to learning the unknown constraints.