What can associative learning do for planning?

Johan Lind

doi:10.1098/rsos.180778

Abstract

There is a new associative learning paradox. The power of associative learning for producing flexible behaviour in non-human animals is downplayed or ignored by researchers in animal cognition, whereas artificial intelligence research shows that associative learning models can beat humans in chess. One phenomenon in which associative learning often is ruled out as an explanation for animal behaviour is flexible planning. However, planning studies have been criticized and questions have been raised regarding both methodological validity and interpretations of results. Due to the power of associative learning and the uncertainty of what causes planning behaviour in non-human animals, I explored what associative learning can do for planning. A previously published sequence learning model which combines Pavlovian and instrumental conditioning was used to simulate two planning studies, namely Mulcahy & Call 2006 ‘Apes save tools for future use.’ Science 312, 1038–1040 and Kabadayi & Osvath 2017 ‘Ravens parallel great apes in flexible planning for tool-use and bartering.’ Science 357, 202–204. Simulations show that behaviour matching current definitions of flexible planning can emerge through associative learning. Through conditioned reinforcement, the learning model gives rise to planning behaviour by learning that a behaviour towards a current stimulus will produce high value food at a later stage; it can make decisions about future states not within current sensory scope. The simulations tracked key patterns both between and within studies. It is concluded that one cannot rule out that these studies of flexible planning in apes and corvids can be completely accounted for by associative learning. Future empirical studies of flexible planning in non-human animals can benefit from theoretical developments within artificial intelligence and animal learning.

Highlights

To the amazement of the world, associative learning models used in artificial intelligence (AI) research achieve human level skills in video games [1] and beat human masters in the Chinese board game& 2018 The Authors
The aim of this study was to explore if a learning model [19], similar to reinforcement learning used in AI research, can help us understand the acquisition of planning behaviour in corvids and apes, behaviours sometimes perceived as complex and human-like
Simulations of the two planning studies on ravens and great apes suggest that behaviour previously claimed to have been generated by flexible planning [24,28] can be accounted for by associative learning

Summary

Introduction

To the amazement of the world, associative learning models used in artificial intelligence (AI) research achieve human level skills in video games [1] and beat human masters in the Chinese board game& 2018 The Authors. It is an intriguing paradox that associative learning is acknowledged for producing complex flexible behaviour within AI research, but is often dismissed and neglected as a model for flexible behaviour in biological systems (both humans and non-human animals). Whether the development of behaviour sequences in non-human animals can be understood in terms of associative learning or not has far-reaching consequences for our understanding of the study of behaviour. If behaviour perceived as advanced or complex, such as chimpanzee (Pan troglodytes) tool use, can develop through associative processes, species differences can be sought in terms of genetic differences in behaviour repertoires, exploratory tendencies such as curiosity, and motivational and attentional factors. It is common that such studies assume only the simplest forms of associative learning This is likely to result in false rejections of associative learning hypotheses. This is because most vertebrates and invertebrates exhibit capacities for both instrumental and Pavlovian learning [21,22], that together with specialized memories [23] make most animals capable of more complex learning than what the simplest forms of associative learning allow

Objectives

Methods

Results

Conclusion