Self-accelerated Thompson sampling with near-optimal regret upper bound

Abstract

Thompson sampling utilizes Bayesian heuristic strategy to balance the exploration-exploitation trade-off. It has been applied in a variety of practical domains and achieved great success. Despite being empirically efficient and powerful, Thompson sampling has eluded theoretical analysis. Existing analyses of Thompson sampling only provide regret upper bound of O˜(d3/2T) for linear contextual bandits, which is worse than the information-theoretic lower bound by a factor of d. In this paper, we design and analyze self-accelerated Thompson Sampling algorithm for the stochastic contextual multi-armed bandit problem. Our analysis establishes that the regret upper bound of self-accelerated Thompson sampling is O˜(dT), which is the best upper bound achieved by any efficient contextual bandit algorithm in infinite action space. Our experiment on simulated data and real-world dataset shows that self-accelerated Thompson sampling outperforms standard Thompson sampling in both convergence rate and prediction accuracy.