Selectable and Unselectable Sets of Neurons in Recurrent Neural Networks With Saturated Piecewise Linear Transfer Function

Abstract

The concepts of selectable and unselectable sets are proposed to describe some interesting dynamical properties of a class of recurrent neural networks (RNNs) with saturated piecewise linear transfer function. A set of neurons is said to be selectable if it can be co-unsaturated at a stable equilibrium point by some external input. A set of neurons is said to be unselectable if it is not selectable, i.e., such set of neurons can never be co-unsaturated at any stable equilibrium point regardless of what the input is. The importance of such concepts is that they enable a new perspective of the memory in RNNs. Necessary and sufficient conditions for the existence of selectable and unselectable sets of neurons are obtained. As an application, the problem of group selection is discussed by using such concepts. It shows that, under some conditions, each group is a selectable set, and each selectable set is contained in some group. Thus, groups are indicated by selectable sets of the RNNs and can be selected by external inputs. Simulations are carried out to further illustrate the theory.