Analog Hopfield Research Articles

Role of asymmetry and external noise in the development and synchronization of oscillations in the analog Hopfield neural networks with time delay.

The analog Hopfield neural network with time delay and random connections has been studied for its similarities in activity to human electroencephalogram and its usefulness in other areas of the applied sciences such as speech recognition, image analysis, and electrocardiogram modeling. Our goal here is to understand the mechanisms that affect the rhythmic activity in the neural network and how the addition of a Gaussian noise contributes to the network behavior. The neural network studied is composed of ten identical neurons. We investigated the excitatory and inhibitory networks with symmetric (square matrix) and asymmetric (triangular matrix) connections. The differential equations that model the network are solved numerically using the stochastic second-order Runge-Kutta method. Without noise, the neural networks with symmetric and asymmetric matrices possessed different synchronization properties: fully connected networks were synchronized both in time and in amplitude, while asymmetric networks were synchronized in time only. Saturation outputs of the excitatory neural networks do not depend on the time delay, whereas saturation oscillation amplitudes of inhibitory networks increase with the time delay until the steady state. The addition of the Gaussian noise is shown to significantly amplify small-amplitude oscillations, dramatically accelerates the rate of amplitude growth to saturation, and changes synchronization properties of the neural network outputs.

Synchronization, Multiresonance Phenomena, and Discrete Oscillation Periods in a Hopfield Neural Network with Two Time Delays

Hopfield neural network attracts particular attention as it serves as a relatively simple mathematical model that describes some properties of the brain function. We investigate analog Hopfield neural networks with two time delays. It is shown that the neural network with all inhibitory connections demonstrates growing oscillations after exceeding the threshold, and oscillations become synchronous after a relatively short period of time of the order of the larger time delay. The oscillation amplitude of the neural network as function of the time delay in one subnetwork demonstrates resonance-like phenomena with multiple peaks. The oscillation period of the neural network with two time delays shows discrete structure and changes within relatively narrow intervals in contrast to the oscillation period of the neural network with one time delay, which shows continuous changes of oscillation amplitude and period with time delay. Multiresonance behavior is sustained for both inhibitory and excitatory connections of the second subnetwork, while the first subnetwork possesses only inhibitory connections. The oscillation period of the subnetwork with smaller time delay and equal coupling strengths dominates the whole neural network activity. Turning on and off smaller time delay in one subnetwork allows for the control of the oscillation period and demonstrates memory-like behavior. Mechanisms of the simulated phenomena are disclosed and their similarity to brain function is discussed.

Self-Correcting Neural Network for Stereo-matching Problem Solving

In the present paper we describe innovative architecture of artificial neural network based on Hopfield structure Self Correcting Neural Network (SCNN). It is implementation similar to dual mode Hopfield-like network for solving stereo matching problem. Considered network consists of basic layer of neurons implemented as analogue Hopfield-like network and supervising layer. Thanks to the supervising layer, there is a possibility of modification of the connection weights between the neurons in the basic layer. This enables the improvement of the network performance (accuracy). Authors propose a depth map use for image segmentation and objects auto-selection. High enough accuracy of these processes can be achieved when proposed network (SCNN) is applied. Similar idea can be applied also for images noise removal. In the present article we also describe in detail neurons dynamics in the basic and supervising layers of the SCNN. The network considered here was a subject of experimental tests using real stereo pictures as well as simulated stereo images. This enabled calculation of error and direct comparison with classic analogue Hopfield neural network.

Improving Wishart Classification of Polarimetric SAR Data Using the Hopfield Neural Network Optimization Approach

This paper proposes the optimization relaxation approach based on the analogue Hopfield Neural Network (HNN) for cluster refinement of pre-classified Polarimetric Synthetic Aperture Radar (PolSAR) image data. We consider the initial classification provided by the maximum-likelihood classifier based on the complex Wishart distribution, which is then supplied to the HNN optimization approach. The goal is to improve the classification results obtained by the Wishart approach. The classification improvement is verified by computing a cluster separability coefficient and a measure of homogeneity within the clusters. During the HNN optimization process, for each iteration and for each pixel, two consistency coefficients are computed, taking into account two types of relations between the pixel under consideration and its corresponding neighbors. Based on these coefficients and on the information coming from the pixel itself, the pixel under study is re-classified. Different experiments are carried out to verify that the proposed approach outperforms other strategies, achieving the best results in terms of separability and a trade-off with the homogeneity preserving relevant structures in the image. The performance is also measured in terms of computational central processing unit (CPU) times.

A novel hybrid-maximum neural network in stereo-matching process.

In the present paper, the completely innovative architecture of artificial neural network based on Hopfield structure for solving a stereo-matching problem—hybrid neural network, consisting of the classical analog Hopfield neural network and the Maximum Neural Network—is described. The application of this kind of structure as a part of assistive device for visually impaired individuals is considered. The role of the analog Hopfield network is to find the attraction area of the global minimum, whereas Maximum Neural Network is finding accurate location of this minimum. The network presented here is characterized by an extremely high rate of work performance with the same accuracy as a classical Hopfield-like network, which makes it possible to use this kind of structure as a part of systems working in real time. The network considered here underwent experimental tests with the use of real stereo pictures as well as simulated stereo images. This enables error calculation and direct comparison with the classic analog Hopfield neural network as well as other networks proposed in the literature.

A Hopfield Neural Network for combining classifiers applied to textured images

In this paper we propose a new method for combining simple classifiers through the analogue Hopfield Neural Network (HNN) optimization paradigm for classifying natural textures in images. The base classifiers are the Fuzzy clustering (FC) and the parametric Bayesian estimator (BP). An initial unsupervised training phase determines the number of clusters and estimates the parameters for both FC and BP. Then a decision phase is carried out, where we build as many Hopfield Neural Networks as the available number of clusters. The number of nodes at each network is the number of pixels in the image which is to be classified. Each node at each network is initially loaded with a state value, which is the membership degree (provided by FC) that the node (pixel) belongs to the cluster associated to the network. Each state is later iteratively updated during the HNN optimization process taking into account the previous states and two types of external influences exerted by other nodes in its neighborhood. The external influences are mapped as consistencies. One is embedded in an energy term which considers the states of the node to be updated and the states of its neighbors. The other is mapped as the inter-connection weights between the nodes. From BP, we obtain the probabilities that the nodes (pixels) belong to a cluster (network). We define these weights as a relation between states and probabilities between the nodes in the neighborhood of the node which is being updated. This is the classifier combination, making the main finding of this paper. The proposed combined strategy based on the HNN outperforms the simple classifiers and also classical combination strategies.

Heat Flux Estimation in Nonlinear Materials Using Kalman Filter-Enhanced Neural Network

This paper presents an efficient technique for analyzing the surface heatflux of a space shuttle upon reentry using a nonlinear inverse heat conduction technique based upon a Kalman filter-enhanced Bayesian backpropagation neural network. The continuous-time analog Hopfield neural network is used to solve various forward problems to obtain training data for the Kalman filter-enhanced Bayesian backpropagation neural network. The calibrated Kalman filter-enhanced Bayesian backpropagation neural network is then used to inversely compute the boundary conditions from given sets of temperature data obtained using the continuous-time analog Hopfield neural network. The results show that the proposed method can predict the unknown parameters of the current inverse problems with an accuracy of 0.001%. The performance of the Kalman filter-enhanced Bayesian backpropagation neural network scheme is shown to be better than that of a Bayesian backpropagation neural network or a stand-alone backpropagation scheme calibrated using a Levenberg–Marquardt backpropagation algorithm.

Applying Neural Networks to the Solution of the Inverse Heat Conduction Problem in a Gun Barrel

The primary cause of gun barrel erosion is the heat generated by the shell as its travels along the barrel. Therefore, calculating the heat flux input to the gun bore is very important when investigating wear problems in the gun barrel and examining its thermomechanical properties. This paper employs the continuous-time analog Hopfield neural network (CHNN) to compute the temperature distribution in various forward heat conduction problems. An efficient technique is then proposed for the solution of inverse heat conduction problems using a three-layered backpropagation neural network (BPN). The weak generalization capacity of BPN networks when applied to the solution of nonlinear function approximations is improved by employing the Bayesian regularization algorithm. The CHNN scheme is used to calculate the temperature in a 155mm gun barrel and the trained BPN is then used to estimate the heat flux of the inner surface of the barrel. The results show that the proposed neural network analysis method successfully solves forward heat conduction problems and is capable of predicting the unknown parameters in inverse problems with an acceptable error.

Solving the Temperature Distribution Field in Nonlinear Heat Conduction Problems Using the Hopfield Neural Network

This article employs the continuous-time analog Hopfield neural network (CHNN) to compute the temperature distribution in nonlinear heat conduction problems. The relationship between the CHNN synaptic connection weights and the governing equations of the nonlinear heat conduction problems is established and a corresponding network connectivity circuit design scheme proposed. The CHNN algorithm is used to solve the heat equation for conduction problems with a power-law nonlinearity. The results confirm that the proposed CHNN scheme provides an accurate means of solving the transient temperature distributions of nonlinear heat conduction problems on a real-time basis.

Solution of inverse heat conduction problems using Kalman filter-enhanced Bayesian back propagation neural network data fusion

This paper presents an efficient technique for analyzing inverse heat conduction problems using a Kalman Filter-enhanced Bayesian Back Propagation Neural Network (KF-B 2PNN). The training data required for the KF-B 2PNN are prepared using the Continuous-time analogue Hopfield Neural Network and the performance of the KF-B 2PNN scheme is then examined in a series of numerical simulations. The results show that the proposed method can predict the unknown parameters in the current inverse problems with an acceptable error. The performance of the KF-B 2PNN scheme is shown to be better than that of a stand-alone Back Propagation Neural Network trained using the Levenberg–Marquardt algorithm.

Applying the Hopfield Neural Network to the Solution of the Temperature Distribution Field in Heat Conduction Problems

This article employs the continuous-time analog Hopfield neural network (CHNN) to compute the temperature distribution in one- and two-dimensional transient heat conduction problems. The relationship between the CHNN synaptic connection weights and the governing equations of the problems is established and a corresponding network connectivity circuit design scheme proposed. The CHNN algorithm is initially applied to the solution of conventional problems and is then used to solve more complicated problems involving time-varying heat flux profiles. The results confirm that the CHNN scheme provides an accurate means of solving the transient temperature distributions of heat conduction problems on a real-time basis.

A Hopfield Neural Network for Image Change Detection

This paper outlines an optimization relaxation approach based on the analog Hopfield neural network (HNN) for solving the image change detection problem between two images. A difference image is obtained by subtracting pixel by pixel both images. The network topology is built so that each pixel in the difference image is a node in the network. Each node is characterized by its state, which determines if a pixel has changed. An energy function is derived, so that the network converges to stable states. The analog Hopfield's model allows each node to take on analog state values. Unlike most widely used approaches, where binary labels (changed/unchanged) are assigned to each pixel, the analog property provides the strength of the change. The main contribution of this paper is reflected in the customization of the analog Hopfield neural network to derive an automatic image change detection approach. When a pixel is being processed, some existing image change detection procedures consider only interpixel relations on its neighborhood. The main drawback of such approaches is the labeling of this pixel as changed or unchanged according to the information supplied by its neighbors, where its own information is ignored. The Hopfield model overcomes this drawback and for each pixel allows a tradeoff between the influence of its neighborhood and its own criterion. This is mapped under the energy function to be minimized. The performance of the proposed method is illustrated by comparative analysis against some existing image change detection methods.

Applying neural networks to the solution of forward and inverse heat conduction problems

This paper employs the continuous-time analogue Hopfield neural network to compute the temperature distribution in forward heat conduction problems and solves inverse heat conduction problems by using a back propagation neural (BPN) network to identify the unknown boundary conditions. The weak generalization capacity of BPN networks is improved by employing the Bayesian regularization algorithm. The feasibility of the proposed method is examined in a series of numerical simulations. The results show that the proposed neural network analysis method successfully solves forward heat conduction problems and is capable of predicting the unknown parameters in inverse problems with an acceptable error.

A nonlinear image reconstruction technique for ECT using a combined neural network approach

A combined multilayer feed-forward neural network (MLFF-NN) and analogue Hopfield network is developed for nonlinear image reconstruction of electrical capacitance tomography (ECT). The (nonlinear) forward problem in ECT is solved using the MLFF-NN trained with a set of capacitance data from measurements based on a back-propagation training algorithm with regularization. The inverse problem is solved using an analogue Hopfield network based on a neural-network multi-criteria optimization image reconstruction technique (HN-MOIRT). The nonlinear image reconstruction based on this combined MLFF-NN + HN-MOIRT approach is tested on measured capacitance data not used in training to reconstruct the permittivity distribution. The performance of the technique is compared against commonly used linear Landweber and semi-linear image reconstruction techniques, showing superiority in terms of both stability and quality of reconstructed images.

On the computational complexity of binary and analog symmetric hopfield nets

We investigate the computational properties of finite binary- and analog-state discrete-time symmetric Hopfield nets. For binary networks, we obtain a simulation of convergent asymmetric networks by symmetric networks with only a linear increase in network size and computation time. Then we analyze the convergence time of Hopfield nets in terms of the length of their bit representations. Here we construct an analog symmetric network whose convergence time exceeds the convergence time of any binary Hopfield net with the same representation length. Further, we prove that the MIN ENERGY problem for analog Hopfield nets is NP-hard and provide a polynomial time approximation algorithm for this problem in the case of binary nets. Finally, we show that symmetric analog nets with an external clock are computationally Turing universal.

Another K-winners-take-all analog neural network

An analog Hopfield type neural network is given, that identifies the K largest components of a list of d of N real numbers. The neurons are identical, with a tanh characteristic, and the weight matrix is symmetric and fully filled. The list to be processed is a summand of the input currents of the neurons, and the network is started from zero. We provide easily computable restrictions on the parameters. The main emphasis here is on the magnitude of the neuronal gain. A complete mathematical analysis is given. The trajectories are shown to eventually have positive components precisely in the positions given by the K largest elements in the input list.

Hopfield network with constraint parameter adaptation for overlapped shape recognition

In this paper, we propose an energy formulation for homomorphic graph matching by the Hopfield network and a Lyapunov indirect method-based learning approach to adaptively learn the constraint parameter in the energy function. The adaptation scheme eliminates the need to specify the constraint parameter empirically and generates valid and better quality mappings than the analog Hopfield network with a fixed constraint parameter. The proposed Hopfield network with constraint parameter adaptation is applied to match silhouette images of keys and results are presented.

A neural approach for obstacle detection with a linear stereoscopic sensor

A neural implementation for achieving real-time obstacle detection in front of a moving vehicle using a linear stereoscopic sensor is presented. The key problem is the so-called “correspondence problem” which consists in matching features in two stereo images that are projections of the same physical entity in the three-dimensional world. In our approach, the set of edge points extracted from each linear image is first split into two classes. Within each of these classes, the matching problem is turned into an optimization task where an energy function, which represents the constraints on the solution, is to be minimized. The optimization problem is then performed thanks to an analog Hopfield neural network. The preliminary discrimination of the edge points allows us to implement the matching process as two networks running in parallel. Experimental results are presented to demonstrate the effectiveness of the approach for 3-D reconstruction in real traffic conditions.

On stability and equilibria of the M-lattice

Both the analog Hopfield network and the cellular neural network are special cases of the M-lattice system, recently introduced to the signal processing community. We prove that a subclass of the M-lattice is totally stable, This result also applies to the original cellular neural network as a rigorous proof of its total stability. By analyzing the stability of fixed points, we derive the conditions for driving the equilibrium outputs of another subclass of the M-lattice to binary values. For the cellular neural network, this analysis is a precise formulation of an earlier argument based on circuit diagrams. And for certain special cases of the analog Hopfield network, this analysis explains why the output variables converge to binary values even with nonzero neuron auto-connections. This behavior, observed in computer simulation by researchers for quite some time, is explained for the first time here.

Solving a kind of nonlinear programming problems via analog neural networks

A high-order analog Hopfield neural network is incorporated to get optimal or near-optimal solutions to a kind of nonlinear programming problems. By employing the operational transconductance amplifier (OTA) and MOS transistors, the practical analog implementation of the neural network can be achieved. The convergence characteristics are demonstrated by computer simulations and two sufficient conditions for the proposed analog neural network to obtain the Lyapunov stable solutions are given.