Vector Memory Research Articles

Cellular neural networks for associative memories

A synthesis procedure for designing nonsymmetric cellular neural networks (CNN) with a predetermined local interconnection structure that will store a set of desired bipolar vectors as memory points is presented. A specific case of constructing Chinese characters is presented to demonstrate the applicability of the results. Simulation results show that all the vectors corresponding to 50 commonly used Chinese characters are reachable memory vectors of the synthesized CNN. >

Letter shortening of the settling time of digital IIR-filters

One of the most serious problems when using IIR-Filters is the behaviour of the settling which depends on the incoming signal and the values which are in the state vector or memory of the filter. The description of the state vector is given in the z-domain for the second direct form and for the cascaded structure. If the incoming signal is monochromatic, and when magnitude, phase and frequency are given or can be calculated, it is possible to determine the state vector (memory) of the filter in order that the settling time of a response to a step change of the monochromatic signal is zero. Various results are shown for one case when the preinitialization is done exactly with no noise present and for other cases when the monochromatic signal is overlaid by white noise.

Solving Linear Systems on Vector and Shared Memory Computers.

Vector and parallel processing overview of current high-performance computers implementation details and overhead performance - analysis, modeling and measurements building blocks in linear algebra direct solution of sparse linear systems iterative solution of sparse linear systems. Appendices: acquiring mathematical software information on various high-performance computers level 1,2, and 3 BLAS quick reference operation counts for various BLAS and decompositions.

Item and associative information in a distributed memory model

According to TODAM, a distributed memory model based on convolution and correlation, item and associative information are stored and retrieved from a common memory vector. The model also gives a possible representation of forgetting, encoding, attentional, and habituation processes. The main purpose of the present paper is to present the necessary derivations so the model can be applied to item recognition, pair recognition, and cued recall in a single-trial or list-learning paradigm. Applications to the list-strength effect, item- and pair-recognition, and paired-associative learning are briefly discussed.

Optical implementation of an improved Hopfield-like retrieval algorithm

A modified calculation of the synaptic matrix for Hopfield-like networks is proposed. By considering the action of such networks in terms of correlation and reconstruction, the modified algorithm simplifies the optical implementation (removing the need for bipolar optical signals) and improves the performance with similar memory vectors. Experimental results, obtained using the new algorithm, are presented, confirming its effectiveness.

Exact associative neural memory dynamics utilizing Boolean matrices

The exact dynamics of shallow loaded associative neural memories are generated and characterized. The Boolean matrix analysis approach is employed for the efficient generation of all possible state transition trajectories for parallel updated binary-state dynamic associative memories (DAMs). General expressions for the size of the basin of attraction of fundamental and oscillatory memories and the number of oscillatory and stable states are derived for discrete synchronous Hopfield DAMs loaded with one, two, or three even-dimensionality bipolar memory vectors having the same mutual Hamming distances between them. Spurious memories are shown to occur only if the number of stored patterns exceeds two in an even-dimensionality Hopfield memory. The effects of odd- versus even-dimensionality memory vectors on DAM dynamics and the effects of memory pattern encoding on DAM performance are tested. An extension of the Boolean matrix dynamics characterization technique to other, more complex DAMs is presented.

D+D 2 Quasiclassical rate constant calculations on parallel computers

The calculation of rate constant values of theH+H 2 reaction for an extended range of excited vibrational states of the diatomic molecule and temperatures is relevant to the modeling ofH − sources. To investigate the effect of isotopic substitutions on the efficiency of vibrational deexcitation processes, we extended the calculations to theD+D 2 system. These calculations were carried out using a program restructured to run on a shared memory vector and parallel computer. The dependence of the efficiency of vibrational deexcitation processes from both the initial vibrational state and temperature of reactants is reported. Restructuring strategies adopted for implementing the program on both shared and distributed memory computers as well as speedups achieved on both types of machines are also discussed.

Linear predictive speech coding arrangement

A speech analyzer is adapted to produce speech parameter signals from a Pth order autocorrelation analysis of a speech pattern in a digital signal processor. The analyzer includes a plurality of memories of predetermined arrangement to store feature vector signals used in the analysis, a single set of coded signals for controlling the analysis, and a memory address processor for addressing the feature vector signal memories. In each iteration of the analysis, at least one speech parameter signal is produced by the digital signal processor responsive to the same set of control signals and the feature vector memory addressing signals.

Speech recognition apparatus

First Page

A theoretical investigation into the performance of the Hopfield model

An analysis is made of the behavior of the Hopfield model as a content-addressable memory (CAM) and as a method of solving the traveling salesman problem (TSP). The analysis is based on the geometry of the subspace set up by the degenerate eigenvalues of the connection matrix. The dynamic equation is shown to be equivalent to a projection of the input vector onto this subspace. In the case of content-addressable memory, it is shown that spurious fixed points can occur at any corner of the hypercube that is on or near the subspace spanned by the memory vectors. Analysed is why the network can frequently converge to an invalid solution when applied to the traveling salesman problem energy function. With these expressions, the network can be made robust and can reliably solve the traveling salesman problem with tour sizes of 50 cities or more.

A feature model of immediate memory

A feature model of immediate memory is presented, and simulations are described. List items are characterized as multiattribute vectors that can be selectively overwritten by subsequent external events and by the ongoing stream of internal activity. Degraded primary memory vectors are compared with intact secondary memory vectors, and retrieval likelihood is computed as the ratio of similarities. The model is shown to account for the major modality-based phenomena of the immediate serial recall literature, including modality-based temporal grouping effects and the negative effects of phonological similarity.

Mathematical aspects of outer-product asynchronous content-addressable memories

We have given evidence by mathematical analysis and by example that the construction of CAM's as quasineural networks based on the twin principles of an outer-product weight matrix and a random asynchronous single-neuron dynamics encounters two obstacles to good performance which appear to be inherent. It is desirable to have the stored memory vectors of a CAM as mutually far apart as possible in order to have unambiguous retrieval with as large a fraction of initial errors (minrad) as possible. For a given number,m, of memory vectors to be stored this requires that their dimension,n, be larger thanm and that the memory vectors be nearly mutually orthogonal (except for complementary pairs). In HOCAM's, it does not seem possible to have both a large minrad and an efficient ratiom/n. Attempts to increasem/n are likely to introduce extraneous fixed-points which reduce minrad appreciably. We have demonstrated this phenomenon in several cases for a particular mode of constructing CAM's of arbitrary size which have a desirable spacing between memory vectors. We conjecture that it is present also in HOCAM's having a random selection of memory vectors. (A mathematical proof of this conjecture now seems possible.) This may account for the rather pessimistic results on capacity obtained by mathematical analysis here and in Cottrell (1988), by a probabilistic analysis in Posner (1987) and by simulation in Hopfield (1982). Further, in Cottrell (1988) there is evidence that outer-product weights are near optimal with respect to minrad, so that otherW may not improve matters. We have left to another paper a study of other approaches to content-addressable memories of which we are aware, but which are not focused on asynchronous dynamics; e.g. computer CAM's as in Kohonen (1977) and biological memory models as in Little (1974); Palm (1980) and Little and Shaw (1978). We have not considered the learning, or adaptive, aspects of CAM's. However, insofar as learning is Hebbian and leads to outer-product weights, our analysis has implications for the effectiveness of learned weights, as may be inferred from our results on ambiguous retrieval.

Consistent parameter estimation in adaptive control for MIMO systems

The problem of consistent parameter estimation in adaptive regulation and adaptive control for multivariable, deterministic systems is addressed. The parameter estimation schemes considered are the RLS estimator, and the RLS estimator with covariance resetting. A decaying excitation technique, similar to that given by Chen and Guo (1987), is used to ensure consistent parameter estimation for the multivariable system without imposing any restrictions on the memory vector φ(k). Some restrictions on the system are relaxed.

On the capacity of associative memories with linear threshold functions

Some important features of various known constructions of associative memories based on linear threshold functions are analyzed. Two important features are dealt with: (a) the ability to select an arbitrary set of desired memory vectors and design a network for this set; (b) the sizes and shapes of the domains of attraction of the desired memory vectors and their relation to various design parameters. The static capacity for randomly chosen desired memories is also analyzed. Two extremal examples of sets of desired memories are then analyzed in detail. For spectral schemes with randomly chosen O(N/ln N) memories, it is shown that almost all of the Hamming sphere around each memory is directly attracted.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Nonlinear network optimization on a massively parallel connection machine

Optimization problems with network constraints arise in several instances in engineering, management, statistical and economic applications. The (usually) large size of such problems motivated research in designing efficient algorithms and software for this problem class. The introduction of parallelism in the design of computer systems adds a new element of complexity to the field. This paper describes the implementation of a distributed relaxation algorithm for strictly convex network problems on a massively parallel computer. A Connection Machine CM-1 configured with 16,384 processing elements serves as the testbed of the implementation. We report computational results with a series of stick percolation and quadratic transportation problems. The algorithm is compared with an implementation of the primal truncated Newton on an IBM 3081-D mainframe, an Alliant FX/8 shared memory vector multiprocessor and the IBM 3090-600 vector supercomputer. One of the larger test problems with approximately 2500 nodes and 8000 arcs requires 1.5 minutes of CPU time on the vector supercomputer. The same problem is solved using relaxation on the CM-1 in less that a second.

Expectation and variance of item resemblance distributions in a convolution-correlation model of distributed memory

The present paper extends the applicability of distributed memory models that use convolution and correlation as their encoding and retrieval operations by providing analytic derivations of the expressions needed to apply such models to a wide range of experimental paradigms. These extensions allow such models to predict recall and recognition performance for single items and double and triple associations under optimal storage and retrieval conditions as well as in situations of degraded probe information at time of test or incomplete encoding of information at time of study. Part I gives a brief review of one convolution-correlation memory model, TODAM (Murdock (1982). Psychological Review, 89, 609–626; TODAM (Murdock (1983). Psychological Review, 90, 316–338). Part II provides the derivations for expectations and variances of the distributions of dot-product resemblance between an extensive list of memory vectors and probe vectors. Expressions of the first two moments of such resemblance distributions in terms of the model parameters are needed to derive model predictions for such dependent measures as recall or recognition accuracy or d′. Part III illustrates the use these resemblance distribution moments by deriving predictions for the data of three experimental paradigms: (1) recall with partial cues (Tulving & Watkins (1973). American Journal of Psychology, 86, 739–748), (2) recognition of rapidly presented information (Loftus, (1974). Memory and Cognition, 2, 545–548), and (3) recognition in context of a triple association learning task (Clark & Shiffrin (1987). Memory and Cognition, 15, 367–378).

The Impact of Hardware Gather/Scatter on Sparse Gaussian Elimination

Recent vector supercomputers provide vector memory access to “randomly” indexed vectors, whereas early vector supercomputers required contiguously or regularly indexed vectors. This additional capability, known as “hardware gather/scatter,” can be used to great effect in general sparse Gaussian elimination. In this note we present some examples that show the impact of this change in hardware on the choice of algorithms for sparse Gaussian elimination. Common folk wisdom holds that general sparse Gaussian elimination algorithms do not perform well on vector computers. Our numerical results demonstrate that hardware gather/scatter allows general sparse elimination algorithms to outperform algorithms based on a band, envelope, or block structure on such computers.

An analysis of vector startup access delays

In a multiport memory access vector processor such as the CRAY X-MP, the numbering of section (lines) relative to bank numbering affects the delay characteristics of vector memory accesses. The author examines numberings which affect the vector startup delay. It has been shown previously that section numberings now associated with the X-MP-4 reduced the steady-state section delays of randomly conflicting vector accesses, vis-a-vis the X-MP-2 numberings. For the present study, it is shown that these same numberings increase the startup delays of such accesses, thus establishing a tradeoff between numbering protocols.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Vector Access Performance in Parallel Memories Using a Skewed Storage Scheme

The degree to which high-speed vector processors approach their peak performance levels is closely tied to the amount of interference they encounter while accessing vectors in memory. In this paper we present an evaluation of a storage scheme that reduces the average memory access time in a vector-oriented architecture. A skewing scheme is used to map vector components into parallel memory modules such that, for most vector access patterns, the number of memory conflicts is reduced over that observed in interleaved parallel memory systems. Address and data buffers are used locally in each module so that transient nonuniformities which occur in some access patterns do not degrade performance. Previous investigations into skewing techniques have attempted to provide conflict-free access for a limited subset of access patterns. The goal of this investigation is different. The skewing scheme evaluated here does not eliminate all memory conflicts but it does improve the average performance of vector access over interleaved systems for a wide range of strides. It is shown that little extra hardware is required to implement the skewing scheme. Also, far fewer restrictions are placed on the number of memory modules in the system than are present in other proposed schemes.

Monte Carlo Photon Transport On Shared Memory and Distributed Memory Parallel Processors

Parallelized Monte Carlo algorithms for analyzing photon transport in an inertially confined fusion (ICF) plasma are consid ered. Algorithms were developed for shared memory (vector and scalar) and distributed memory (scalar) parallel pro cessors. The shared memory algorithm was implemented on the IBM 3090/400, and timing results are presented for dedi cated runs with two, three, and four pro cessors. Two alternative distributed memory algorithms (replication and dis patching) were implemented on a hyper cube parallel processor (1 through 64 nodes). The replication algorithm yields essentially full efficiency for all cube sizes; with the 64-node configuration, the absolute performance is nearly the same as with the CRAY X-MP The dispatching algorithm also yields efficiencies above 80% in a large simulation for the 64-pro cessor configuration.