Architecture of Associative Memory with Reduced Cross Talk and Its Performance Formulation

Abstract

Listen

Translate article

The extended conventional high order associative memory, including how to apply the input key and to obtain the recalled output, is represented as follows: \({z}={F}\left\{\Sigma_{{i}=1}^{{v}}\left({Y}^{({i})\ \otimes}{X}_{(\alpha)}^{{(i)}}\right).{X}_{(\alpha )}\right\}\), where ⊗ denotes the outer product operator, F denotes the signum function that operates componentwise and \({X}_{(\alpha)}^{{(i)}}\) denotes the i-th extended high order memorized key vector of \(\Sigma_{{k}=0}\alpha\) nCk — length whose components consist of mutually disjoint high order cross product up to α-th order. X(α) is an extended high order input vector constructed under the same condition. Previously, we proposed a new architecture with reduced cross talk, which completely eliminated cross talk due to the memorizing keys having odd number Hamming distance from the input key and furthermore, a relatively large part of the remaining cross talk due to even number Hamming distance from the input key by introducing (N,n) Hamming code into all the memorizing keys and input key, i.e., \({z}={F}\left [ \Sigma_{{i}=1}^{{v}}(1/2)\left \{ \left ({Y}^{({i})\otimes\bar{{X}} _{(\alpha)} ^{{(i)}}} \right).\bar{{X}}_{(\alpha)}+\otimes \overset{\simeq({i})}{{X}}({N}- \alpha).\overset{\simeq}{{X}}({N}- \alpha)\right \} \right ]\) denote the i-th high order memorized keys and input key extended from N to (N->x)-th order, encoded as (N,n) Hamming Code, respectively. The basic idea of the construction of this architecture is based upon the finding that the recalled output z of conventional associative memory, extended to high order correlation, can be represented in an explicit functional form in terms of the Hamming distance d between the memorized keys and input key as follows: \(z = F\{ \sum\limits_{d = 0}^n ( \Sigma \;iv(d \in I^{(d)} y(i_v (d)))H(d,\alpha ;n)\} \), where \({H}({d},\alpha;{n})=\Sigma_{{k}=0}^{\alpha}h({d},{k};{n}),{h}({d,k:n})=\Sigma_{\zeta =0}^{{d}}\)\(\left(-1\right)^{\zeta}\cdot\ {d}^{{C}_{\zeta}}\cdot\ ({n}-{d})^{{C}}(k-\zeta),\ {and}\ I^{({d})}=\left \{i_{{v}}({d});{i}_{{v}}({d})\in\ {I},{v}=1,...,{v}_{{d}}\ {and}(1/2)\cdot\Sigma_{{j}}^{{n}}=0\left | {x}_{{j}}^{{i}_{{v}}({d})}-{x}_{{j}} \right|={d}\right\}\) is a partition of the index set of data I={i: y(i), i=l,…,v}. In order to investigate the performance characteristics of conventional and the proposed architecture, computer experiments were performed by using randomly generated bipolar data and keys. To make the comparison on cost performance basis, the architectures considered use a nearly equal number of cross products though the order of correlation is different in each case. The proposed architectures require less order of correlation compared to the conventional architecture and this makes the construction of the proposed architecture less tedious. In this paper, to investigate quantitatively the ability of exact recall of the proposed architecture, we formularize the probability of exact data recall as the functional form of Hamming distance between the memorized keys and the input key under the assumption that all the memorizing keys and data are subjected to be mutually statistically independent. When the desired datum to be recalled exactly is y(i*), the computational result is as shown below: $$Prob(z = y^{(1*)} ) = \{ \frac{1}{{\sqrt {2\pi } }}\int_{ - \infty }^{H(0,\alpha ;N/)} {\sqrt {\sum\limits_{d \in \{ d:d = 0\bmod .2\} } {(\frac{{v - 1}}{{2^n }})\cdot\omega (d,N)\cdot H(d,\alpha ;N)^2 } } } {\text{exp(- }}\frac{{u^2 }}{2})du\} ^m ,$$ where \(\omega({d,N}) ={N}^{C}{d}+\Sigma_{\gamma=0}^{{d}}(-1)^{\gamma.}(({N}-1)/2){C}({d}-\gamma)^.((N+1)/2)^{C}{\gamma}\) Experimental results show that the theoretical estimation of the probability of exact data recall is in well agreement with the experimental results even though the parameters n, N, m and v used in the experiments are not so large. In this paper, the percentage of data recall by the architecture was considered on all the exactly recalled data and also those data in which all the components except for only one is correctly recalled by using (15,11) Hamming Code for the memorized data. In this experiment, the proposed architecture achieved excellent recall (98.9 % by theoretical estimation and 99.2% by experimental result with 500 memorized pairs), and signified that, by employing error correcting code in all data of length 11, almost all the memorized data can be exactly recalled. Both the theoretical and experimental results strongly demonstrate the superior ability of the proposed architecture, especially the one with Hamming Code and substantiate the effective reduction of cross talk.