Modeling the Impact of Habituation and Breaks in Exploitation Process on Multi-Armed Bandits Performance

Abstract

Habituation is a common phenomenon in learning, where the response to a repeated stimulus decreases over time. In Multi-Armed Bandits (MAB) algorithms used for visual content delivery optimization, habituation can lead to suboptimal performance. For example, if an agent becomes habituated to a suboptimal arm, it may continue to choose that arm even if better options are available. Habituation can be modeled as a form of ”forgetting”, where the agent gradually loses confidence in its estimates of the reward probabilities for each arm as time passes. Proposed approach allows updating estimates according to habituation model, while also exploiting the arm with the highest estimated reward probability. The results showed that it is a very good solution to introduce breaks in the multi-armed bandit habit model, which was implemented and tested in our study.