Chapter 6 - Finite State Machines and Recurrent Neural Networks — Automata and Dynamical Systems Approaches

Abstract

The chapter presents two approaches to analyze the relationship between a recurrent neural network (RNN) and the finite state machine “M” that the network is able to exactly mimic. In the first approach, the network is treated as a state machine, and the relationship between the RNN and “M” is established in the context of the algebraic theory of automata. In the second approach, the RNN is viewed as a set of discrete-time dynamical systems associated with input symbols of M. The chapter presents issues concerning network representation of loops and cycles in the state transition of “M” (as shown to provide a basis for the interpretation of learning process from the point of view of bifurcation analysis) and studies the relationship between RNN and automata. This relationship between an RNN and a finite state machine that it exactly mimics has been investigated from two points of view; first, when the network is treated as a state machine; and second, when the RNN is viewed as a set of discrete-time dynamical systems associated with input symbols of “M.”