Complex-valued Hopfield Neural Network Research Articles

In-Memory Computing Circuit Implementation of Complex-Valued Hopfield Neural Network for Efficient Portrait Restoration

Complex-valued neural networks have better optimization capabilities, stronger robustness and richer characterization capabilities compared with real-valued neural networks, which has achieved good results in the field of portrait restoration. However, there is almost no circuit implementation of complex-valued neural networks. Based on this, this paper proposes an in-memory computing circuit implementation of complex-valued Hopfield neural network (CHNN) for the first time, which provides a highly accurate and efficient processing circuit for portrait restoration. Firstly, a new memristive array is proposed, which can realize parallel complex-valued multiplication and complex-valued vector-matrix multiplication. On the basis, a CHNN circuit that can perform large-scale recursive computations is designed. Due to the characteristics of in-memory computation, the computation speed and robustness have been improved when realizing portrait restoration. Different portrait restoration scenarios can be realized based on the programmability of the memristive array. Pspice simulation results show that the recovery speed of CHNN can reach the level of 0.1 ms, and the accuracy can reach above 97.00%. Robustness analysis shows that the circuit can tolerate a certain degree of programming error and has strong anti-noise performance.

Storage Capacity of Quaternion-Valued Hopfield Neural Networks With Dual Connections.

A complex-valued Hopfield neural network (CHNN) is a multistate Hopfield model. A quaternion-valued Hopfield neural network (QHNN) with a twin-multistate activation function was proposed to reduce the number of weight parameters of CHNN. Dual connections (DCs) are introduced to the QHNNs to improve the noise tolerance. The DCs take advantage of the noncommutativity of quaternions and consist of two weights between neurons. A QHNN with DCs provides much better noise tolerance than a CHNN. Although a CHNN and a QHNN with DCs have the samenumber of weight parameters, the storage capacity of projection rule for QHNNs with DCs is half of that for CHNNs and equals that of conventional QHNNs. The small storage capacity of QHNNs with DCs is caused by projection rule, not the architecture. In this work, the ebbian rule is introduced and proved by stochastic analysis that the storage capacity of a QHNN with DCs is 0.8 times as many as that of a CHNN.

Hyperbolic-valued Hopfield neural networks in hybrid mode

A complex-valued Hopfield neural network (CHNN) has been applied as a multistate neural associative memory. Although synchronous mode accelerates the recall process, a CHNN with a projection rule may be trapped at a cycle of length two. A hyperbolic-valued Hopfield neural network (HHNN) with a conventional projection rule converges to a fixed point in synchronous mode. A noise robust projection rule improves the noise tolerance of HHNN, though it is not able to be applied to an HHNN in synchronous mode. In this work, we proposed hybrid mode, that is, asynchronous mode after synchronous mode. The conventional projection rule is employed in synchronous mode, and the noise robust projection rule is employed in asynchronous mode. Thus, the HHNN in hybrid mode converges in both modes. The HHNN in hybrid mode is expected to provide better noise tolerance than an HHNN in synchronous mode and faster recall than an HHNN in asynchronous mode. Computer simulations imply that our expectations are achieved.

Quaternion Projection Rule for Rotor Hopfield Neural Networks.

A rotor Hopfield neural network (RHNN) is an extension of a complex-valued Hopfield neural network (CHNN) and has excellent noise tolerance. The RHNN decomposition theorem says that an RHNN decomposes into a CHNN and a symmetric CHNN. For a large number of training patterns, the projection rule for RHNNs generates large self-feedbacks, which deteriorates the noise tolerance. To remove self-feedbacks, we propose a projection rule using quaternions based on the decomposition theorem. Using computer simulations, we show that the quaternion projection rule improves noise tolerance.

Complex-valued Hopfield neural networks with real weights in synchronous mode

A complex-valued Hopfield neural network with a multistate activation function converges in asynchronous mode, whereas it converges or is trapped at a cycle of length two in synchronous mode. For parallel processing, convergence in synchronous mode is necessary. In particular, a complex-valued Hopfield neural network (CHNN) is the most widely used multistate Hopfield model, and it is desirable that a CHNN converges in synchronous mode. In this work, a real-weight CHNN (RWCHNN) is proposed. An RWCHNN restricts the weights of a CHNN to real numbers and has half weight parameters of a CHNN. We provide the stability conditions of an RWCHNN in synchronous mode and prove that the projection rule satisfies them. The RWCHNN is the first model such that it converges in synchronous mode and has half weight parameters of a CHNN. Computer simulations show that the average update count until convergence is far less than those of the hypercomplex-valued Hopfield neural networks with the same number of weight parameters of an RWCHNN.

Hyperbolic Hopfield neural networks for image classification in content-based image retrieval

Neural networks play a significant role in data classification. Complex-valued Hopfield Neural Network (CHNN) is mostly used in various fields including the image classification. Though CHNN has proven its credibility in the classification task, it has a few issues. Activation function of complex-valued neuron maps to a unit circle in the complex plane affecting the resolution factor, flexibility and compatibility to changes, during adaptation in retrieval systems. The proposed work demonstrates Content-Based Image Retrieval System (CBIR) with Hyperbolic Hopfield Neural Networks (HHNN), an analogue of CHNN for classifying images. Activation function of the Hyperbolic neuron is not cyclic in hyperbolic plane. The images are mathematically represented and indexed using the six basic features. The proposed HHNN classifier is trained, tested and evaluated through extensive experiments considering individual features and four combined features for indexing. The obtained results prove that HHNN guides retrieval process, enhances system performance and minimizes the cost of implementing Neural Network Classifier-based image retrieval system.

Synthesis of complex- and hyperbolic-valued Hopfield neural networks

A complex-valued Hopfield neural network (CHNN), a multistate Hopfield model, is useful for processing multilevel data, such as image data. Several alternatives of CHNN have been proposed. A hyperbolic-valued Hopfield neural network (HHNN) improves the noise tolerance of CHNN. In this work, we propose a synthetic Hopfield neural network (SHNN), a combination of a CHNN and an HHNN. An SHNN is the first combination of Hopfield models using different algebras. Since a CHNN and an HHNN have different operators, such as addition and multiplication, they are represented by matrices and vectors to compose an SHNN. A CHNN and an HHNN have different global minima. Only the common global minima are the global minima of SHNN. Thus, an SHNN is expected to improve the noise tolerance. In fact, computer simulations support our expectation.

Noise-Robust Projection Rule for Rotor and Matrix-Valued Hopfield Neural Networks.

A complex-valued Hopfield neural network (CHNN) has weak noise tolerance due to rotational invariance. Some alternatives of CHNN, such as a rotor Hopfield neural network (RHNN) and a matrix-valued Hopfield neural network (MHNN), resolve rotational invariance and improve the noise tolerance. However, the RHNN and MHNN with projection rules have a different problem of self-feedbacks. If the self-feedbacks are reduced, the noise tolerance is expected to be improved further. For reduction in the self-feedbacks, the noise-robust projection rules are introduced. The stability conditions are extended, and the self-feedbacks are reduced based on the extended stability conditions. Computer simulations support that the noise tolerance is improved. In particular, the noise tolerance is more robust against an increase in the number of training patterns.

Bicomplex Projection Rule for Complex-Valued Hopfield Neural Networks.

A complex-valued Hopfield neural network (CHNN) with a multistate activation function is a multistate model of neural associative memory. The weight parameters need a lot of memory resources. Twin-multistate activation functions were introduced to quaternion- and bicomplex-valued Hopfield neural networks. Since their architectures are much more complicated than that of CHNN, the architecture should be simplified. In this work, the number of weight parameters is reduced by bicomplex projection rule for CHNNs, which is given by the decomposition of bicomplex-valued Hopfield neural networks. Computer simulations support that the noise tolerance of CHNN with a bicomplex projection rule is equal to or even better than that of quaternion- and bicomplex-valued Hopfield neural networks. By computer simulations, we find that the projection rule for hyperbolic-valued Hopfield neural networks in synchronous mode maintains a high noise tolerance.

Diagonal rotor Hopfield neural networks

A complex-valued Hopfield neural network (CHNN) is a multistate Hopfield model and has been applied to the storage of image data. It has the weak noise tolerance due to the inherent property of rotational invariance. A hyperbolic-valued Hopfield neural network (HHNN) resolves rotational invariance and improves the noise tolerance. A rotor Hopfield neural network (RHNN) is an extension of CHNN and the weights are represented by matrices. It provides excellent noise tolerance by resolving the rotational invariance. However, an RHNN needs double weight parameters of a CHNN, unlike an HHNN. In this work, we propose a diagonal RHNN (DRHNN), which restricts the weights of RHNN to diagonal matrices and reduces the number of weight parameters. The number of weight parameters in a DRHNN, an HHNN, and a CHNN is same. In addition, a DRHNN resolves the rotational invariance and provides excellent noise tolerance like an RHNN.

A Delay-Dividing Approach to Robust Stability of Uncertain Stochastic Complex-Valued Hopfield Delayed Neural Networks

In scientific disciplines and other engineering applications, most of the systems refer to uncertainties, because when modeling physical systems the uncertain parameters are unavoidable. In view of this, it is important to investigate dynamical systems with uncertain parameters. In the present study, a delay-dividing approach is devised to study the robust stability issue of uncertain neural networks. Specifically, the uncertain stochastic complex-valued Hopfield neural network (USCVHNN) with time delay is investigated. Here, the uncertainties of the system parameters are norm-bounded. Based on the Lyapunov mathematical approach and homeomorphism principle, the sufficient conditions for the global asymptotic stability of USCVHNN are derived. To perform this derivation, we divide a complex-valued neural network (CVNN) into two parts, namely real and imaginary, using the delay-dividing approach. All the criteria are expressed by exploiting the linear matrix inequalities (LMIs). Based on two examples, we obtain good theoretical results that ascertain the usefulness of the proposed delay-dividing approach for the USCVHNN model.

Matrix-valued twin-multistate Hopfield neural networks

A complex-valued Hopfield neural network (CHNN) has been widely used for the storage of image data. The CHNN has been extended using hypercomplex numbers. A couple of hypercomplex-valued Hopfield neural networks employ a twin-multistate activation function to reduce the numbers of weight parameters. In this work, we propose a matrix-valued twin-multistate Hopfield neural network (MTMHNN), whose neuron states and weights are 2 × 2 matrices. Computer simulations show that the MTMHNN has better noise tolerance than the hypercomplex-valued twin-multistate Hopfield neural networks.

New complex-valued activation functions

We present a new type of activation functions for a complex-valued neuralnetwork (CVNN). A proposed activation function is constructed such that itfixes a given ellipse. We obtain an application to a complex-valued Hopfieldneural network (CVHNN) using a special form of the introduced complexfunctions as an activation function. Considering the interesting geometricproperties of the plane curve ellipse such as focusing property, weemphasize that these properties may have possible applications in variousneural networks.

Storage capacity of hyperbolic Hopfield neural networks

A complex-valued Hopfield neural network (CHNN) is a multistate model of Hopfield neural network, and has been applied to the storage of multi-level data. A hyperbolic Hopfield neural network (HHNN) is another multistate model. The storage capacity is an important issue of Hopfield neural networks. In this work, the storage capacity of HHNN is analyzed, and is compared with that of CHNN.

Noise Robust Projection Rule for Hyperbolic Hopfield Neural Networks.

A complex-valued Hopfield neural network (CHNN) is a multistate Hopfield model. Low noise tolerance is the main disadvantage of CHNNs. The hyperbolic Hopfield neural network (HHNN) is a noise robust multistate Hopfield model. In HHNNs employing the projection rule, noise tolerance rapidly worsened as the number of training patterns increased. This result was caused by the self-loops. The projection rule for CHNNs improves noise tolerance by removing the self-loops, however, that for HHNNs cannot remove them. In this brief, we extended the stability condition for the self-loops of HHNNs and modified the projection rule. Thus, the HHNNs had improved noise tolerance.

Twin-multistate commutative quaternion Hopfield neural networks

Abstract The complex-valued Hopfield neural network (CHNN) can deal with multi-level information, and has often been applied to the storage of image data. The quaternion Hopfield neural network (QHNN) is a multistate model of a Hopfield neural network, and requires half the connection weight parameters of CHNN. In this study, we propose a commutative quaternion Hopfield neural network (CQHNN) as the analogy of QHNN. The multiplication of commutative quaternions is commutative and convenient, unlike that of quaternions. We compared the noise tolerance of CQHNNs and QHNNs by computer simulation, and discuss the simulation results from the perspective of rotational invariance and self loops.

Storage capacity of rotor Hopfield neural networks

Abstract Hopfield neural networks have been studied by many researchers. A complex-valued Hopfield neural network (CHNN) is a multistate model of Hopfield neural network, and has been applied to the storage of multilevel data, such as image data. A rotor Hopfield neural network (RHNN) is an extension of CHNN. The RHNNs demonstrated double the storage capacity of CHNNs and excellent noise tolerance by computer simulations. Jankowski et al. analyzed the storage capacity of CHNNs by approximating the crosstalk term using central limit theorem. In this work, we show that the RHNNs have double the storage capacity of the CHNNs based on their theory.

Complex-Valued Associative Memories with Projection and Iterative Learning Rules

Abstract In this paper, we investigate the stability of patterns embedded as the associative memory distributed on the complex-valued Hopfield neural network, in which the neuron states are encoded by the phase values on a unit circle of complex plane. As learning schemes for embedding patterns onto the network, projection rule and iterative learning rule are formally expanded to the complex-valued case. The retrieval of patterns embedded by iterative learning rule is demonstrated and the stability for embedded patterns is quantitatively investigated.

Fixed points of symmetric complex-valued Hopfield neural networks

Abstract A complex-valued Hopfield neural network (CHNN) is a model of a multistate Hopfield neural network, and has been applied to the storage of multilevel data. Weak noise tolerance, however, is a disadvantage of CHNNs. Symmetric CHNNs (SCHNNs), modified CHNNs, improve the noise tolerance of CHNNs. In the present work, we study the global and local minima of SCHNNs with one training pattern. In CHNNs, the global minima are the training and rotated patterns, and there are no local minima. In SCHNNs, it has been hard to determine all the global and local minima. It is thought that the global minima are the training and reversed patterns, and that there are no local minima in most cases. In the present work, however, we find many local minima, and show that they are very weak attractors, which reduce noise tolerance very little. In addition, we determine all the global minima of SCHNNs.

Decomposition of Rotor Hopfield Neural Networks Using Complex Numbers.

A complex-valued Hopfield neural network (CHNN) is a multistate model of a Hopfield neural network. It has the disadvantage of low noise tolerance. Meanwhile, a symmetric CHNN (SCHNN) is a modification of a CHNN that improves noise tolerance. Furthermore, a rotor Hopfield neural network (RHNN) is an extension of a CHNN. It has twice the storage capacity of CHNNs and SCHNNs, and much better noise tolerance than CHNNs, although it requires twice many connection parameters. In this brief, we investigate the relations between CHNN, SCHNN, and RHNN; an RHNN is uniquely decomposed into a CHNN and SCHNN. In addition, the Hebbian learning rule for RHNNs is decomposed into those for CHNNs and SCHNNs.