Adapting to the task environment: Explorations in expected value

Abstract

Small variations in how a task is designed can lead humans to trade off one set of strategies for another. In this paper we discuss our failure to model such tradeoffs in the Blocks World task using ACT-R’s default mechanism for selecting the best production among competing productions. ACT-R’s selection mechanism, its expected value equation, has had many successes (see, for example [Anderson, J. R., & Lebiere, C. (Eds.). (1998). Atomic components of thought. Hillsdale, NJ: Lawrence Erlbaum Associates.]) and a recognized strength of this approach is that, across a wide variety of tasks, it tends to produce models that adapt to their task environment about as fast as humans adapt. (This congruence with human behavior is in marked contrast to other popular ways of computing the utility of alternative choices; for example, Reinforcement Learning or most Connectionist learning methods.) We believe that the failure to model the Blocks World task stems from the requirement in ACT-R that all actions must be counted as a binary success or failure. In Blocks World, as well as in many other circumstances, actions can be met with mixed success or partial failure. Working within ACT-R’s expected value equation we replace the binary success/failure judgment with three variations on a scalar one. We then compare the performance of each alternative with ACT-R’s default scheme and with the human data. We conclude by discussing the limits and generality of our attempts to replace ACT-R’s binary scheme with a scalar credit assignment mechanism.