Getting symbols out of a neural architecture

Abstract

Traditional connectionist networks are sharply limited as general accounts of human perception and cognition because they are unable to represent relational ideas such as loves (John, Mary) or bigger-than (Volkswagen, breadbox) in a way that allows them to be manipulated as explicitly relational structures. This paper reviews and critiques the four major responses to this problem in the modelling community: (1) reject connectionism (in any form) in favour of traditional symbolic approaches to modelling the mind; (2) reject the idea that mental representations are symbolic (i.e. reject the idea that we can represent relations); and (3) attempt to represent symbolic structures in a connectionist/neural architecture by finding a way to represent role–filler bindings. Approach (3) is further subdivided into (3a) approaches based on varieties of conjunctive coding and (3b) approaches based on dynamic role–filler binding. I will argue that (3b) is necessary to get symbolic processing out of a neural computing architecture. Specifically, I will argue that vector addition is both the best way to accomplish dynamic binding and an essential part of the proper treatment of symbols in a neural architecture.