Adaptive learning via selectionism and Bayesianism, Part I: Connection between the two

Abstract

According to the selection-by-consequence characterization of operant learning, individual animals/species increase or decrease their future probability of action choices based on the consequence (i.e., reward or punishment) of the currently selected action (the so-called "Law of Effect"). Under Bayesianism, on the other hand, evidence is evaluated based on likelihood functions so that action probability is modified from a priori to a posteriori according to the Bayes formula. Viewed as hypothesis testing, a selectionist framework attributes evidence exclusively to the selected, focal hypothesis, whereas a Bayesian framework distributes across all hypotheses the support from a piece of evidence. Here, an intimate connection between the two theoretical frameworks is revealed. Specifically, it is proven that when individuals modify their action choices based on the selectionist's Law of Effect, the learning population, on the ensemble level, evolves according to a Bayesian-like dynamics. The learning equation of the linear operator model [Bush, R. R., & Mosteller, F. (1955). Stochastic models for learning, New York: John Wiley and Sons], under ensemble averaging, yields the class of predictive reinforcement learning models (e.g., [Busemeyer, J. R., & Myung, I. J. (1992). An adaptive approach to human decision making: Learning theory, decision theory, and human performance. Journal of Experimental Psychology: General, 121, 177-194; Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16, 1936-1947]).