M-ary Hopfield neural network for storage and retrieval of variable length sequences: Multi-limit cycle approach

Abstract

In this paper, we present M-ary Hopfield neural network based auto-associative memory formulation for variable length sequences. Each sequence is composed of variable length of multi-valued pattern vectors. Recently, we presented M-ary Hopfield neural network for storage and retrieval of fixed length sequences. In this paper, we focus on two important issues (i) storage and retrieval mechanism for variable length pattern sequences in a single network (ii) analysis of storage capacity of the network for variable length sequences. We use Pseudo-Inverse learning based dual weight learning for storing variable length sequences as multiple limit cycles in a single M-ary Hopfield neural network. We propose a novel recall mechanism based on energies of End-of-Sequence markers to retrieve the stored sequence (complete sequence) and sequence id, corresponding to the trigger state. We present results of experiments on a large dataset of variable length sequences constructed from movie clips (i) to establish the efficacy of the proposed retrieval mechanism for sequence retrieval under zero error tolerance, given any particular state of the sequence as trigger state to the network, (ii) to demonstrate trade off between number of cycles and average length of cycle which the network supports for storage and retrieval (iii) to demonstrate that the system retains O(N) storage capacity for variable length sequences modelled in its limit cycles. This is attempted for the first time in the area of attractor neural networks.