A survey on contextual multi-armed bandits

Abstract

As a powerful reinforcement learning framework, Contextual Multi-Armed Bandits have extensive applications in various domains. The models of Contextual Multi-Armed Bandits enable decision-makers to make intelligent choices in situations with uncertainty, and they find utility in fields such as online advertising, medical treatment optimization, resource allocation, and more. This paper reviews the evolution of algorithms for Contextual Multi-Armed Bandits, including traditional Bayesian approaches and the latest deep learning techniques. Successful case studies are summarized in different application domains, such as online ad click-through rate optimization and medical decision support. Furthermore, the author discusses future research directions, including more sophisticated context modeling, interpretability, fairness issues, and ethical considerations in the context of automated decision-making.