Hopfield Associative Memory Research Articles

Quantum fluctuation dynamics of open quantum systems with collective operator-valued rates, and applications to Hopfield-like networks

We consider a class of open quantum many-body systems that evolves in a Markovian fashion, the dynamical generator being in GKS-Lindblad form. Here, the Hamiltonian contribution is characterized by an all-to-all coupling, and the dissipation features local transitions that depend on collective, operator-valued rates, encoding average properties of the system. These types of generators can be formally obtained by generalizing, to the quantum realm, classical (mean-field) stochastic Markov dynamics, with state-dependent transitions. Focusing on the dynamics emerging in the limit of infinitely large systems, we build on the exactness of the mean-field equations for the dynamics of average operators. In this framework, we derive the dynamics of quantum fluctuation operators, that can be used in turn to understand the fate of quantum correlations in the system. We then apply our results to quantum generalized Hopfield associative memories. Here we show that, asymptotically and at the description level of quantum fluctuations, only a very weak amount of quantum correlations, in the form of quantum discord, emerges beyond classical correlations.

Toward Optimal Synthesis of Discrete-Time Hopfield Neural Network.

In this research brief, the relationship between eigenvectors (with {+1, -1} components) of a synaptic weight matrix W and the stable/anti-stable states of discrete-time Hopfield associative memory (HAM) is established. Also, the synthesis of W with desired stable/anti-stable states using spectral representation of W in even/odd dimension is discussed when the threshold vector is a non-zero vector. Freedom in choice of eigenvalues is capitalized to improve the noise immunity of the Hopfield neural network (HNN). Also, the problem of optimal synthesis of Hopfield Associative memory is formulated.

Mean-field dynamics of open quantum systems with collective operator-valued rates: validity and application

We consider a class of open quantum many-body Lindblad dynamics characterized by an all-to-all coupling Hamiltonian and by dissipation featuring collective ‘state-dependent’ rates. The latter encodes local incoherent transitions that depend on average properties of the system. This type of open quantum dynamics can be seen as a generalization of classical (mean-field) stochastic Markov dynamics, in which transitions depend on the instantaneous configuration of the system, to the quantum domain. We study the time evolution in the limit of infinitely large systems, and we demonstrate the exactness of the mean-field equations for the dynamics of average operators. We further derive the effective dynamical generator governing the time evolution of (quasi-) local operators. Our results allow for a rigorous and systematic investigation of the impact of quantum effects on paradigmatic classical models, such as quantum generalized Hopfield associative memories or (mean-field) kinetically-constrained models.

A quantum Hopfield associative memory implemented on an actual quantum processor

In this work, we present a Quantum Hopfield Associative Memory (QHAM) and demonstrate its capabilities in simulation and hardware using IBM Quantum Experience.. The QHAM is based on a quantum neuron design which can be utilized for many different machine learning applications and can be implemented on real quantum hardware without requiring mid-circuit measurement or reset operations. We analyze the accuracy of the neuron and the full QHAM considering hardware errors via simulation with hardware noise models as well as with implementation on the 15-qubit ibmq_16_melbourne device. The quantum neuron and the QHAM are shown to be resilient to noise and require low qubit overhead and gate complexity. We benchmark the QHAM by testing its effective memory capacity and demonstrate its capabilities in the NISQ-era of quantum hardware. This demonstration of the first functional QHAM to be implemented in NISQ-era quantum hardware is a significant step in machine learning at the leading edge of quantum computing.

A neuro-genetic Hopfield associative memory (HAM) with improved noise immunity

Artificial Neural Network (ANN) is a form of computation inspired by the arrangement and purpose of the brain. Pattern Recognition is one of the demanding areas under Artificial Neural Network and pattern association is one of the major tasks through which pattern recognition is realized. Nowadays researchers are taking interest in the combination of evolutionary algorithms and ANN. In the present work, an attempt is being made to analyze the performance of Hopfield Associative Memory with Monte Carlo adaptation rule and evolutionary searching (Genetic Algorithm) for pattern storage and recall. The training pattern set has been taken a set of five Greek alphabet letters as different objects. The aim is to store Patterns with MC adaptation rule and get the best weight matrices for efficient recalling of any given set patterns. The neural networks architectures have been taken up for storing the given set of inputs, for which conventional Hebbian and Monte Carlo adaptation rule is considered. During the mutation process, various mutation probabilities are supposed to be evaluated. The detailed simulated results thus obtained will be presented with the help of tables and graph.

Prospects for the implementation of neural network technologies in radar information processing systems

Conducted studies of the phased implementation of neural network technologies in the practice of processing radar information, providing for a gradual increase in the level of neural network methods in processing systems, have shown that the use of neural network technologies can improve the quality of radar information processing in the most difficult conditions that require high computing power, when the dynamics of changes in external conditions is very is high and traditional approaches to the creation of processing systems are not able to provide the required level of efficiency. The need to develop theoretical provisions for neural network processing of radar information was revealed, while the main features of information processing in radars determine the relevance of research devoted to preventing the reduction in the quality of radar images in conditions of a large number of targets and a complex «jamming» environment based on the rational use of neural network technology. Analysis of the phased implementation of neural network technologies in radar information processing systems, as well as the use of neural network technology for processing radar information in terms of search and research, makes it possible to increase the efficiency of neural network methods for all processing tasks. Assessment of the required performance of computational tools allows us to single out the main neural network paradigms, the use of which gives a tangible increase in the efficiency of radar information processing, such as multilayer perceptron, Hopfield associative memory and self-organizing Kohonen network, while it is possible to rank the proposed methods in accordance with the required performance, undemanding to computing power and implemented on existing or promising computing facilities with software implementation of neural network paradigms. The analysis of possible directions for improving the quality of radar information processing does not claim to fully cover the entire multifaceted area of such studies. In this paper, only the most universal and widespread neural network paradigms are considered and the main part of possible areas of their application is analyzed. However, the proposed options show that the use of neural network technologies in critical tasks will improve the efficiency of radar information processing for complex, rapidly changing external conditions. The use of the principles of self-learning and the developed apparatus for the synthesis of neural network methods will reduce the duration and complexity of theoretical research, the conduct of which is a necessary and mandatory part of the traditional approach. In the course of further research, some of the proposed methods can be refined, as well as the emergence of new methods that make it possible to more fully use the advantages of neural network technology. Carrying out further research work in these areas will give a powerful stimulating impetus for the creation in the future of highly efficient methods for processing radar information, which can be implemented on the available element base.

Signatures of Associative Memory Behavior in a Multimode Dicke Model.

Dicke-like models can describe a variety of physical systems, such as atoms in a cavity or vibrating ion chains. In equilibrium these systems often feature a radical change in their behavior when switching from weak to strong spin-boson interaction. This usually manifests in a transition from a "dark" to a "superradiant" phase. However, understanding the out-of-equilibrium physics of these models is extremely challenging, and even more so for strong spin-boson coupling. Here we show that the nonequilibrium strongly interacting multimode Dicke model can mimic some fundamental properties of an associative memory-a system which permits the recognition of patterns, such as letters of an alphabet. Patterns are encoded in the couplings between spins and bosons, and we discuss the dynamics of the spins from the perspective of pattern retrieval in associative memory models. We identify two phases, a "paramagnetic" and a "ferromagnetic" one, and a crossover behavior between these regimes. The "ferromagnetic" phase is reminiscent of pattern retrieval. We highlight similarities and differences with the thermal dynamics of a Hopfield associative memory and show that indeed elements of "machine learning behavior" emerge in the strongly coupled multimode Dicke model.

Influence of random topology in artificial neural networks: A survey

Abstract Due to the fully-connected complex structure of Artificial Neural Networks (ANNs), systems based on ANN may consume much computational time, energy and space. Therefore, intense research has been recently centered on changing the topology and design of ANNs to obtain high performance. To explore the influence of network structure on ANNs complex systems topologies have been applied in these networks to have more efficient and less complex structures while they are more similar to biological systems at the same time. In this paper, the methodology and results of some recent papers are summarized and discussed in which the authors investigated the efficacy of random complex networks on the performance of Hopfield associative memory and multi-layer ANNs compared with ANNs with small-world, scale-free and regular structures.

Nontrivial Attractors of the Perturbed Nonlinear Schrödinger Equation: Applications to Associative Memory and Pattern Recognition

AbstractComputers are dynamical systems that carry out information processing through their change of state with time. For instance, neural networks such as Hopfield's associative memory are dissipative dynamical systems in a finite dimensional configuration space with attractors that represent stored patterns. In particular, dissipative dynamical systems with an infinite dimensional configuration space are of broad interest and have the possibility to store and restore complex and strongly distorted data structures. In this work, the nonlinear Schrödinger equation (NLSE) with a dissipative perturbation which creates a frictional force acting on soliton is considered. It is shown that the control of the perturbative term allows one to decrease the velocity of soliton to zero and conserve a positive value of its amplitude. The existence of such a frictional force would suggest that the perturbation makes the zero‐velocity solitonic solution of non‐zero amplitude into an attractor for all evolution trajectories whose initial conditions are moving solitons. This allows a way to store information in the infinite dimensional dynamical system governed by NLSE using principles which are completely analogous to that of Hopfield's associative memory. This approach can be realized with solitons in Bose–Einstein condensates and nonlinear optical systems.

High storage capacity in the Hopfield model with auto-interactions—stability analysis

Recent studies point to the potential storage of a large number of patterns in the celebrated Hopfield associative memory model, well beyond the limits obtained previously. We investigate the properties of new fixed points to discover that they exhibit instabilities for small perturbations and are therefore of limited value as associative memories. Moreover, a large deviations approach also shows that errors introduced to the original patterns induce additional errors and increased corruption with respect to the stored patterns.

Quantum annealing with ultracold atoms in a multimode optical resonator

A dilutely filled $N$-site optical lattice near zero temperature within a high-$Q$ multimode cavity can be mapped to a spin ensemble with tailorable interactions at all length scales. The effective full site to site interaction matrix can be dynamically controlled by the application of up to $N(N+1)/2$ laser beams of suitable geometry, frequency and power, which allows for the implementation of quantum annealing dynamics relying on the all-to-all effective spin coupling controllable in real time. Via an adiabatic sweep starting from a superfluid initial state one can find the lowest energy stationary state of this system. As the cavity modes are lossy, errors can be amended and the ground state can still be reached even from a finite temperature state via ground state cavity cooling. The physical properties of the final atomic state can be directly and almost non-destructively read off from the cavity output fields. As example we simulate a quantum Hopfield associative memory scheme.

Chaotic pseudo-orthogonalized Hopfield associative memory

A Hopfield associative memory (HAM) is a major model of associative memory using neural networks. The classical HAM model has a low storage capacity. A pseudo-orthogonalized HAM (POHAM) was therefore proposed to improve storage capacity by encoding the training patterns to be pseudo-orthogonal. In the present work, we examine a different property of POHAM. A chaotic HAM (CHAM) can explore embedded patterns, including training patterns. Although it is not desirable to recall patterns other than training patterns, at a minimum, the reversed patterns are recalled. A POHAM regards the training and reversed patterns as equivalent. To take advantage of this property, we propose a chaotic POHAM (CPOHAM). We evaluated a CHAM and a CPOHAM using computer simulations. The CPOHAM never recalled the reversed patterns. In addition, the CPOHAM recalled very few other pseudo-memories, such as mixture patterns, due to the orthogonality of the encoded training patterns, while the CHAM frequently recalled the mixture patterns.

Chaotic complex‐valued bipartite auto‐associative memory with a periodic activation function

A Hopfield associative memory (HAM) can store binary data. A HAM has been extended to a complex‐valued HAM (CHAM) which can store multi‐level data. A CHAM has often been applied to the storage of grayscale images. A CHAM has a problem of rotational invariance, which makes many psuedomemories and reduces the noise tolerance. A chaotic CHAM can explore the stored patterns, though it recalls many pseudomemories. A complex‐valued bipartite auto‐associative memory (CBAAM) was proposed to improve the noise tolerance by solving rotational invariance. Since a CBAAM stores the reversed patterns, a chaotic CBAAM (CCBAAM) recalls both the training and reversed patterns. In order to solve this problem, a periodic activation function is introduced. A periodic activation function identifies the training and reversed patterns, so that a CCBAAM does not recall the reversed patterns. We evaluate our proposed CCBAAM by computer simulations. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Quantum-Inspired Multidirectional Associative Memory With a Self-Convergent Iterative Learning.

Quantum-inspired computing is an emerging research area, which has significantly improved the capabilities of conventional algorithms. In general, quantum-inspired hopfield associative memory (QHAM) has demonstrated quantum information processing in neural structures. This has resulted in an exponential increase in storage capacity while explaining the extensive memory, and it has the potential to illustrate the dynamics of neurons in the human brain when viewed from quantum mechanics perspective although the application of QHAM is limited as an autoassociation. We introduce a quantum-inspired multidirectional associative memory (QMAM) with a one-shot learning model, and QMAM with a self-convergent iterative learning model (IQMAM) based on QHAM in this paper. The self-convergent iterative learning enables the network to progressively develop a resonance state, from inputs to outputs. The simulation experiments demonstrate the advantages of QMAM and IQMAM, especially the stability to recall reliability.

Automatic welding quality classification for the spot welding based on the Hopfield associative memory neural network and Chernoff face description of the electrode displacement signal features

To develop an automatic welding quality classification method for the spot welding based on the Chernoff face image created by the electrode displacement signal features, an effective pattern feature extraction method was proposed by which the Chernoff face images were converted to binary ones, and each binary image could be characterized by a binary matrix. According to expression categories on the Chernoff face images, welding quality was classified into five levels and each level just corresponded to a kind of expression. The Hopfield associative memory neural network was used to build a welding quality classifier in which the pattern feature matrices of some weld samples with different welding quality levels were remembered as the stable states. When the pattern feature matrix of a test weld is input into the classifier, it can be converged to the most similar stable state through associative memory, thus, welding quality corresponding to this finally locked stable state can represent the welding quality of the test weld. The classification performance test results show that the proposed method significantly improves the applicability and efficiency of the Chernoff faces technique for spot welding quality evaluation and it is feasible, effective and reliable.

Analysis of Hopfield Associative Memory with Combination of MC Adaptation Rule and an Evolutionary Algorithm

Analysis of Hopfield Associative Memory with Combination of MC Adaptation Rule and an Evolutionary Algorithm

DNA and the brain

The idea that artificial neural networks could be based on molecular components is not new, but making such a system has been difficult. A network of four artificial neurons made from DNA has now been created. See Letter p.368 Before neuron-based brains evolved, complex biomolecular circuits must have endowed individual cells with the intelligent behaviour that ensures survival. But the study of how molecules can 'think' has not yet produced useful molecule-based computational systems that mimic even a single neuron. In a study that straddles the fields of DNA nanotechnology, DNA computing and synthetic biology, Qian et al. use DNA as an engineering material to construct computing circuits that exhibit autonomous brain-like behaviour. The team uses a simple DNA gate architecture to create reaction cascades functioning as a 'Hopfield associative memory', which can be trained to 'remember' DNA patterns and recall the most similar one when presented with an incomplete pattern. The challenge now is to use the strategy to design autonomous chemical systems that can recognize patterns or molecular events, make decisions and respond to the environment.

Neural network computation with DNA strand displacement cascades

The impressive capabilities of the mammalian brain--ranging from perception, pattern recognition and memory formation to decision making and motor activity control--have inspired their re-creation in a wide range of artificial intelligence systems for applications such as face recognition, anomaly detection, medical diagnosis and robotic vehicle control. Yet before neuron-based brains evolved, complex biomolecular circuits provided individual cells with the 'intelligent' behaviour required for survival. However, the study of how molecules can 'think' has not produced an equal variety of computational models and applications of artificial chemical systems. Although biomolecular systems have been hypothesized to carry out neural-network-like computations in vivo and the synthesis of artificial chemical analogues has been proposed theoretically, experimental work has so far fallen short of fully implementing even a single neuron. Here, building on the richness of DNA computing and strand displacement circuitry, we show how molecular systems can exhibit autonomous brain-like behaviours. Using a simple DNA gate architecture that allows experimental scale-up of multilayer digital circuits, we systematically transform arbitrary linear threshold circuits (an artificial neural network model) into DNA strand displacement cascades that function as small neural networks. Our approach even allows us to implement a Hopfield associative memory with four fully connected artificial neurons that, after training in silico, remembers four single-stranded DNA patterns and recalls the most similar one when presented with an incomplete pattern. Our results suggest that DNA strand displacement cascades could be used to endow autonomous chemical systems with the capability of recognizing patterns of molecular events, making decisions and responding to the environment.

Reducing Spurious States by Rotor Associative Memory

Hopfield Associative Memory (HAM) is a typical model of associative memories. Complex-valued Associative Memory is a complex-valued version of HAM. They store not only learning patterns but other undesired patterns, which are called spurious states. The reversed patterns in HAM and the rotated patterns in CAM are typical spurious states. We propose to reduce spurious states by using Rotor Associative Memory (RAM). The RAM does not store the rotated patterns except the reversed patterns. So the RAM removes most of spurious states from the CAM. By adding a dummy state, the RAM also can remove them from the HAM. To find spurious states, we construct a searching system similar to the chaotic CAM. By using the searching system, we investigate whether the RAM can reduce the number of spurious states.

Gradient Descent Learning for Rotor Associative Memory

Complex-valued Associative Memory (CAM) is an extended model of Hopfield Associative Memory (HAM). The fundamental elements, such as input-output signals and connection weights of the CAM are extended to complex numbers. The CAM can deal with multi-states information. Rotor Associative Memory (RAM) is an extended model of the CAM. Rotor neurons are essentially equivalent to complex-valued neurons. Connection weights of the RAM are expressed by two by two matrices. Only hebb rule has been proposed for the learning of the RAM. Its storage capacity is small, so advanced learning methods are necessary. In this paper, we propose gradient descent learning rule for the RAM (GDR RAM). It is based on that for the CAM (GDR CAM) proposed by Lee. We solved the learning rule and performed computer simulations to compare the GDR CAM and the GDR RAM. At last, it turned out that the storage capacity of the GDR RAM is approximately twice as much as that of the GDR CAM and the noise robustness of the GDR RAM is much better than that of the GDR CAM.