Learning and Optimization with Seasonal Patterns

Abstract

Decision Making in a Nonstationary Environment with Periodic Rewards Multiarmed bandit (MAB) is a powerful tool in sequential decision making. Traditional MAB models assume constant mean rewards over time, an assumption often too restrictive for real-world applications in which rewards can vary seasonally. In “Learning and Optimization with Seasonal Patterns,” Chen, Wang, and Wang challenge the standard assumption and study a nonstationary MAB model with periodic rewards. They introduce a two-stage policy that combines Fourier analysis with a confidence bound–based learning procedure. This innovative approach allows the algorithm to adapt to time-varying mean rewards that follow a periodic pattern. The first stage estimates the periods of all decision-making arms, whereas the second stage exploits this information to optimize long-term rewards. The study proves that the learning policy is near optimal, achieving a regret upper bound that scales with the square root of the time horizon and the periods of the arms. This work opens new avenues for more adaptive and efficient decision making in many applications that face seasonality, such as the fashion industry and service systems.