Comparing feedforward and recurrent neural network architectures with human behavior in artificial grammar learning

Andrea Alamia,Rufin Vanrullen,Dimitri Paisios,Victor Gauducheau

doi:10.1038/s41598-020-79127-y

Abstract

In recent years artificial neural networks achieved performance close to or better than humans in several domains: tasks that were previously human prerogatives, such as language processing, have witnessed remarkable improvements in state of the art models. One advantage of this technological boost is to facilitate comparison between different neural networks and human performance, in order to deepen our understanding of human cognition. Here, we investigate which neural network architecture (feedforward vs. recurrent) matches human behavior in artificial grammar learning, a crucial aspect of language acquisition. Prior experimental studies proved that artificial grammars can be learnt by human subjects after little exposure and often without explicit knowledge of the underlying rules. We tested four grammars with different complexity levels both in humans and in feedforward and recurrent networks. Our results show that both architectures can “learn” (via error back-propagation) the grammars after the same number of training sequences as humans do, but recurrent networks perform closer to humans than feedforward ones, irrespective of the grammar complexity level. Moreover, similar to visual processing, in which feedforward and recurrent architectures have been related to unconscious and conscious processes, the difference in performance between architectures over ten regular grammars shows that simpler and more explicit grammars are better learnt by recurrent architectures, supporting the hypothesis that explicit learning is best modeled by recurrent networks, whereas feedforward networks supposedly capture the dynamics involved in implicit learning.

Highlights

In recent years artificial neural networks achieved performance close to or better than humans in several domains: tasks that were previously human prerogatives, such as language processing, have witnessed remarkable improvements in state of the art models
For each grammar we assessed whether participants learned the rules by testing accuracy and reaction times (RT) by means of a Bayesian ANOVA, considering BLOCK as independent factor and SUBJECT as a random factor
We found very similar results in both accuracy and RT in analyzing the sub-set of participants who failed in reporting the rules during the questionnaire in context free (CF) and context sensitive (CS) grammars

Summary

Introduction

In recent years artificial neural networks achieved performance close to or better than humans in several domains: tasks that were previously human prerogatives, such as language processing, have witnessed remarkable improvements in state of the art models. A formal definition of grammar complexity (i.e. Chomsky’s hierarchy6) provides a theoretical framework to study grammar learning; on the other hand, previous studies in humans set a well-defined experimental framework to compare human behavior with the performance of different neural network architectures. Experimental results have shown that participants can learn artificial grammars well irrespective of their level in the Chomsky hierarchy This demonstrates that the hierarchy may reflect some form of computational complexity, but it does not reflect cognitive c omplexity[13]; possibly, different computational processes may be involved in human grammar learning. Contrary to human experiments in which subjects typically see a few dozen examples, all considered computational models have been trained with large datasets, and sometimes with significant overlap between training and test sets, making it difficult to draw any substantial comparisons with human cognition

Objectives

Methods

Results

Discussion

Conclusion