Modeling Choice Variation in Search Strategies with Multi-Armed Bandit Problems

Abstract

Prior research in decisions from experience (DFE) involving multi-armed bandit problems has used the sampling paradigm. In this paradigm, decision-makers search for information between multiple options before making a final consequential choice. Prior research in the sampling paradigm has accounted for information search and final choices using computational cognitive models. However, little is known on how cognitive models could account for final choices of participants with different exploration strategies in the presence or absence of an intermediate option. In this paper, we perform an individual-differences analysis and test the ability of computational models to explain final choices of participants with different exploration strategies in the absence or presence of an intermediate option. Specifically, we take an Instance-Based Learning (IBL) model, which relies on recency and frequency processes, and we calibrate this model to final choices of participants exhibiting more-switching (piecewise strategy) or less-switching (comprehensive strategy) between options in different problems. Also, a Natural Mean Heuristic (NMH) model, relying on frequency of experienced outcomes, is used as a baseline. Results revealed that both IBL and NMH models explained aggregate and individual choices better when participants followed piecewise strategy compared to the comprehensive strategy. Overall, the IBL model, calibrated to individual participants using a single set of parameters, performed better compared to the NMH model. We highlight the implications of our results for DFE research involving exploration before consequential decisions.