Computation, Cognition, and Neural Networks

Abstract

Computation and Cognition. To understand computing with neural networks and how cognitive processes can be implemented in this way, it is imperative to distinguish between two kinds of machines and the forms of computation they produce: Finite-state automata (FA’s), and machines such as the Turing machine (TM) or the pushdown automaton (PA). The difference concerns the use of (working) memory. In an FA, the memory is interwoven with the program. To increase processing capacity, new states have to be added to the machine, with new connections between these states and the old states. As a result, the structure of the machine changes and thus the function computed by the machine. In contrast, a TM or a PA is an FA connected to a memory that is external to the FA. The FA is the program of the machine. Because the FA (program) is separated from the memory, the processing capacity (memory) of the machine can be increased without changing the program, and thus without changing the function computed. To use an external memory, a program in a TM or PA has to produce representations (symbols) that can be stored in and retrieved from the memory. This shows why computation of this kind is symbol manipulation: The program manipulates symbols by the processes of storing and retrieving. With a repeated use of storing and retrieving, combinations (strings) of symbols can be stored in the memory, to be used for recognition or production.