Quaternionic Hopfield Neural Networks Research Articles

Multistate vector product hopfield neural networks

Several high-dimensional models of Hopfield neural networks, such as complex-valued and quaternionic Hopfield neural networks, have been proposed. However, it has been hard to construct three-dimensional models of Hopfield neural networks. A split type of vector product Hopfield neural network (VPHNN) was proposed as a special case of ordinary Hopfield neural networks. It is easier to construct split types of Hopfield neural networks than multistate types of ones, because the split types can be often regarded as special cases of ordinary ones. In the present work, we extend the split VPHNN to the multistate VPHNN. We define its energy and a primitive learning algorithm, the Hebbian learning rule. In addition, we prove the stability of multistate VPHNNs. Furthermore, we investigated the fundamentals of multistate VPHNNs, such as the storage capacity and noise tolerance, by computer simulations.

Quaternionic Hopfield neural networks with twin-multistate activation function

In the present work, a Hopfield neural network with a new quaternionic activation function, referred to as a twin-multistate activation function, is proposed. The multistate activation function has been used in complex-valued Hopfield neural networks (CHNNs). It is useful for representing multilevel information, and the CHNNs with a multistate activation function have been applied to the storage of multilevel data, such as gray-scale images. A twin-multistate activation function consists of two multistate activation functions. Quaternionic Hopfield neural networks (QHNNs) with a twin-multistate activation function can take the place of the CHNNs with a multistate activation function. The QHNNs require half the number of connection parameters of CHNNs. Projection rule is a fast learning algorithm, and is suitable under restricted computational power. However, it requires full-connection, and the sparse connections are not allowed. Projection rule is also available for the QHNNs with a twin-multistate activation function. When the memory resource and computational power are restricted, the QHNNs with a twin-multistate activation function is useful. In the present work, the conventional network topology is only considered. To take advantage of non-commutativity of quaternions, several researchers proposed network topology for QHNNs, such as dual connections. In future, the QHNNs with a twin-multistate activation function will be extended to those with such network topology.

On the Dynamics of Hopfield Neural Networks on Unit Quaternions.

In this paper, we first address the dynamics of the elegant multivalued quaternionic Hopfield neural network (MV-QHNN) proposed by Minemoto et al. Contrary to what was expected, we show that the MV-QHNN, as well as one of its variation, does not always come to rest at an equilibrium state under the usual conditions. In fact, we provide simple examples in which the network yields a periodic sequence of quaternionic state vectors. Afterward, we turn our attention to the continuous-valued quaternionic Hopfield neural network (CV-QHNN), which can be derived from the MV-QHNN by means of a limit process. The CV-QHNN can be implemented more easily than the MV-QHNN model. Furthermore, the asynchronous CV-QHNN always settles down into an equilibrium state under the usual conditions. Theoretical issues are all illustrated by examples in this paper.

Pseudo-Orthogonalization of Memory Patterns for Complex-Valued and Quaternionic Associative Memories

Abstract Hebbian learning rule is well known as a memory storing scheme for associative memory models. This scheme is simple and fast, however, its performance gets decreased when memory patterns are not orthogonal each other. Pseudo-orthogonalization is a decorrelating method for memory patterns which uses XNOR masking between the memory patterns and randomly generated patterns. By a combination of this method and Hebbian learning rule, storage capacity of associative memory concerning non-orthogonal patterns is improved without high computational cost. The memory patterns can also be retrieved based on a simulated annealing method by using an external stimulus pattern. By utilizing complex numbers and quaternions, we can extend the pseudo-orthogonalization for complex-valued and quaternionic Hopfield neural networks. In this paper, the extended pseudo-orthogonalization methods for associative memories based on complex numbers and quaternions are examined from the viewpoint of correlations in memory patterns. We show that the method has stable recall performance on highly correlated memory patterns compared to the conventional real-valued method.

Gradient descent learning for quaternionic Hopfield neural networks

A Hopfield neural network (HNN) is a neural network model with mutual connections. A quaternionic HNN (QHNN) is an extension of HNN. Several QHNN models have been proposed. The hybrid QHNN utilizes the non-commutativity of quaternions. It has been shown that hybrid QHNNs with the Hebbian learning rule outperformed QHNNs in noise tolerance. The Hebbian learning rule, however, is a primitive learning algorithm, and it is necessary that we study advanced learning algorithms. Although the projection rule is one of a few promising learning algorithms, it is restricted by network topologies and cannot be applied to hybrid QHNNs. In the present work, we propose gradient descent learning, which can be applied to hybrid QHNNs. We compare the performance of gradient descent learning with that of projection rule. Results showed that the gradient descent learning outperformed projection rule in noise tolerance in computer simulations. For small training-pattern sets, hybrid QHNNs with gradient descent learning produced the best performances. QHNNs did so for large training-pattern set. In future, gradient descent learning will be extended to QHNNs with different network topology and activation function.

Symmetric quaternionic Hopfield neural networks

Quaternion algebra is an extension of complex number system and has the property of non-commutativity. Quaternionic Hopfield neural networks (QHNNs) are extensions of complex-valued Hopfield neural networks (CHNNs) using quaternions, and several models have been proposed. Both the CHNNs and QHNNs have low noise tolerance due to rotational invariance. Recently, a novel CHNN model, a symmetric CHNN (SCHNN), has been proposed to improve noise tolerance of CHNNs. In the present work, this scheme is extended to the QHNNs. The proposed model is referred to as a symmetric QHNN (SQHNN). We show that the SQHNNs improve the noise tolerance by computer simulations.

Fixed points of split quaternionic hopfield neural networks

In a complex-valued phasor Hopfield neural network with a training pattern, only the rotated patterns are fixed points, while in a complex-valued K-state Hopfield neural network, there exist K fixed points. In the case of a quaternionic Hopfield neural network (QHNN) with a continuous activation function, again only the rotated patterns are fixed points. We consider a QHNN with a split activation function, which is a 16-state activation function. This type of QHNN is referred to as a split QHNN (SQHNN). It is expected to have 16 fixed points, all of which are global minima. The rate at which the training pattern would be recalled from random initial states would thus be expected to be 1/16. However, the rate was higher in our computer simulations. We investigate the reasons for this discrepancy.

Rotational invariance of quaternionic hopfield neural networks

High‐dimensional neural networks have been studied by many researchers for their flexible representation. Quaternionic Hopfield neural networks (QHNNs) are one of them. Quaternions have the inherent property of non‐commutative multiplication. Connection weights act differently from left and right in QHNNs, and we can have left QHNNs (LQHNNs) and right QHNNs (RQHNNs). Hybrid QHNNs (HQHNNs), which are the compound models of LQHNNs and RQHNNs, have been proposed, and their excellent noise tolerance has been shown through computer simulations. It has been pointed out that the improvement in their noise tolerance is related to the avoidance of rotational invariance. Thus, it is very important to study rotational invariance. In this paper, we investigate the rotational invariance of QHNNs. Rotational invariance in LQHNNs and RQHNNs is different. We define the left and right‐rotated vectors, which are fixed in RQHNNs and LQHNNs, respectively. From the standpoint of HQHNNs, the common rotated vectors are important. We also study the common rotated vectors and provide a couple of exceptional examples. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

ASSOCIATIVE MEMORY IN QUATERNIONIC HOPFIELD NEURAL NETWORK

Associative memory networks based on quaternionic Hopfield neural network are investigated in this paper. These networks are composed of quaternionic neurons, and input, output, threshold, and connection weights are represented in quaternions, which is a class of hypercomplex number systems. The energy function of the network and the Hebbian rule for embedding patterns are introduced. The stable states and their basins are explored for the networks with three neurons and four neurons. It is clarified that there exist at most 16 stable states, called multiplet components, as the degenerated stored patterns, and each of these states has its basin in the quaternionic networks.