Multi-Armed Bandit Algorithms: Innovations and Applications in Dynamic Environments

Abstract

This paper delves into the fundamental concept of the Multi-Armed Bandit (MAB) problem, structuring its analysis around two primary phases. The initial phase, exploration, is dedicated to investigating the potential rewards of each arm. Subsequently, the exploitation phase utilizes insights from exploration to maximize returns. The discussion then progresses to elucidate the core methodologies and workflows of three principal MAB algorithms: Upper Confidence Bound (UCB), Thompson Sampling, and Epsilon-Greedy. These algorithms are meticulously analyzed for their unique approaches and efficiencies in handling the MAB problem. Expanding the scope further, the paper spotlights three practical applications of MAB algorithms. The first application involves Dynamic Resource Allocation in Multi-Unmanned Aerial Vehicle (UAV) Air-Ground Networks, leveraging the K-armed Bandit framework. This is followed by an exploration of Product Pricing Algorithms grounded in MAB principles, offering innovative solutions for dynamic pricing strategies. Lastly, the paper examines a cost-effective MAB algorithm tailored for dense wireless networks, addressing the complexities and demands of modern network infrastructures. This comprehensive study not only highlights the versatility of MAB algorithms but also underscores their growing importance in diverse real-world applications.