Morphing Consumer Dynamics: Bandits Meet HMM

Abstract

Websites are created to help visitors take some action, such as making a purchase or a donation. As visitors click on various webpages, they may make rapid steps towards the action or may bounce away. Websites that can adapt to match such consumer dynamics perform better. However, assessing visitor’s changing distance to the action, at each click, and adapting to it in real time is challenging because of the sheer number of design elements that are found in websites, that combine exponentially. We solve this problem by matching latent states to webpage designs, combining recent advances in multi-armed bandit literature, website morphing and HMM literature. We develop a novel dynamic program to explicitly model the trade- off firms face between nudging a visitor to later states along the funnel, and maximizing immediate reward given current estimates of purchase probabilities. We use a HMM to assess visitors’ states in real time, and couple it with a MAB model to learn the effectiveness of each design×state combination. We provide a proof-of-concept in two applications. First, we run a field study on MBA website of a major European university. Second, we implement our algorithm on a cloud server and test it on a online store for consumer electronics. In both applications we find that matching morphs to dynamic states using our algorithm outperforms current MAB methods (fixed design) and alternative policies.