Randomly connected sigma–pi neurons can form associator networks

Abstract

A set of sigma–pi units randomly connected to two input vectors forms a type of hetero-associator related to convolution- and matrix-based associative memories. Associations are represented as patterns of activity rather than connection strengths. Decoding the associations requires another network of sigma–pi units, with connectivity dependent on the encoding network. Learning the connectivity of the decoding network involves setting n3 parameters (where n is the size of the vectors), and can be accomplished in approximately 3e n log n presentations of random patterns. This type of network encodes information in activation values rather than in weight values, which makes the information about relationships accessible to further processing. This accessibility is essential for higher-level cognitive tasks such as analogy processing. The fact that random networks can perform useful operations makes it more plausible that these types of associative network could have arisen in the nervous systems of natural organisms during the course of evolution.