Broadening the Tests of Learning Models

Abstract

For many years psychological studies of the learning process have used a simulated medical diagnosis task in which symptom configurations are probabilistically related to diseases. Participants are given a set of symptoms and asked to indicate which disease is present, and feedback is given on each trial. We enrich this standard laboratory task in four different ways. First, the symptoms have four possible values (low, medium low, medium high, and high) rather than just two. Second, symptom configurations are generated from an expanded factorial design rather than a simple factorial design. Third, subjects are asked to make a continuous judgment indicating their confidence in the diagnosis, rather than simply a binary judgment. Fourth, cumulated performance scores, payoffs, and the availability of a historical summary of the outcomes are varied in order to assess how these treatments modulate performance. These enrichments provide a broader data set and more challenging tests of the models. Using 123 subjects each in 480 trials, we compare five existing learning models plus several variants, including the well-known Bayesian, fuzzy logic, connectionist, exemplar, and ALCOVE models. We find that the subjects do learn to distinguish the symptom configurations, that subjects are quite heterogeneous in their response to the task, and that only a small part of the variation across subjects arises from the differences in treatments. The most striking finding is that the model that best predicts subjects' behavior is a simple Bayesian model with a single fitted parameter for prior precision to capture individual differences. We use rolling regression techniques to elucidate the behavior of this model over time and find some evidence of over-response to current stimuli.