Recurrent Hopfield Research Articles

A recurrent Hopfield network for estimating meso-scale effective connectivity in MEG

The architecture of communication within the brain, represented by the human connectome, has gained a paramount role in the neuroscience community. Several features of this communication, e.g., the frequency content, spatial topology, and temporal dynamics are currently well established. However, identifying generative models providing the underlying patterns of inhibition/excitation is very challenging. To address this issue, we present a novel generative model to estimate large-scale effective connectivity from MEG. The dynamic evolution of this model is determined by a recurrent Hopfield neural network with asymmetric connections, and thus denoted Recurrent Hopfield Mass Model (RHoMM). Since RHoMM must be applied to binary neurons, it is suitable for analyzing Band Limited Power (BLP) dynamics following a binarization process.We trained RHoMM to predict the MEG dynamics through a gradient descent minimization and we validated it in two steps. First, we showed a significant agreement between the similarity of the effective connectivity patterns and that of the interregional BLP correlation, demonstrating RHoMM’s ability to capture individual variability of BLP dynamics. Second, we showed that the simulated BLP correlation connectomes, obtained from RHoMM evolutions of BLP, preserved some important topological features, e.g, the centrality of the real data, assuring the reliability of RHoMM.Compared to other biophysical models, RHoMM is based on recurrent Hopfield neural networks, thus, it has the advantage of being data-driven, less demanding in terms of hyperparameters and scalable to encompass large-scale system interactions. These features are promising for investigating the dynamics of inhibition/excitation at different spatial scales.

Distributed leader-follower based adaptive consensus control for networked microgrids

Networked microgrids (NMGs) provide a promising solution for accommodating various distributed energy resources (DERs) and enhance its performance. However, the coordinated operation of the system with integration of a large number of DERs is major challenge. Therefore, this article aims to provide a distributed control strategy (DCS) based on a leader-follower framework for the coordination of multiple DERs in NMGs. A fuzzy optimized Recurrent Hopfield Neural Network (F-HNN) designed self-adaptive fractional order proportional integral derivative (FOPID) controller is proposed for distributed frequency control of NMGs. Initially, a Lyapunov-based objective function is derived for weight updation of the proposed network. The fuzzy approach is used to optimize the output of the HNN based on its gradients. The proposed F-HNN based DCS is implemented in MATLAB/Simulink and validated for frequency regulation of NMGs through hardware-in-the-loop simulation (HIL) using OPAL-RT. The results obtained are compared with the conventional FOPID HNN tuned and other classical controls. The self-adaptiveness of the controller is demonstrated for change in renewable power generation. Furthermore, the resiliency of the controller is tested with communication failures and, plug and play operation of MGs. The results obtained showed that the frequency of the NMG system is well regulated within the band of ±0.2 Hz. Also, the transient and steady state performance reveals that the proposed DCS is more significant than other techniques.

English

Rice crop yield prediction suggests the procedure of estimating the rice quantity which is harvested in a provided land region dependent upon several features like farming practices, weather conditions, and soil quality. The main aim of rice crop yield prediction is to offer farmers and agricultural planners correct crop yield calculations in progress, creating informed decisions assuming harvesting, marketing, and planting their crops. It supports farmers in optimizing their production and enhancing their profitability, but also improving food security by ensuring an even supply of rice for consumers. Deep learning (DL) approaches are utilized for predicting crop yield by leveraging the influence of neural networks for learning complex patterns and connections in data. This study presents a Teaching and Learning Based Optimization with Deep Learning for Rice Crop Yield Prediction (TLBODL-RCYP) technique. The proposed TLBODL-RCYP approach emphasizes the accurate forecasting of the rice yield using DL and hyperparameter optimizers. To accomplish this, the TLBODL-RCYP technique performs different preprocessing stages to improve the data quality. Besides, the TLBODL-RCYP technique employs a hybrid Convolution Recurrent HopField Neural Network (HCRHNN) model for yield prediction. At last, the TLBO algorithm was utilized to adjust the hyperparameter values of the HCRHNN technique and thereby enhance the predictive results. The experimental outcome investigation of the TLBODL-RCYP approach is tested with the Kaggle dataset, and the outcomes assured the improvized predictive results of the TLBODL-RCYP method over other recent DL techniques.

Modified 2 Satisfiability Reverse Analysis Method via Logical Permutation燨perator

The effectiveness of the logic mining approach is strongly correlated to the quality of the induced logical representation that represent the behaviour of the data. Specifically, the optimum induced logical representation indicates the capability of the logic mining approach in generalizing the real datasets of different variants and dimensions. The main issues with the logic extracted by the standard logic mining techniques are lack of interpretability and the weakness in terms of the structural and arrangement of the 2 Satisfiability logic causing lower accuracy. To address the issues, the logical permutation serves as an alternative mechanism that can enhance the probability of the 2 Satisfiability logical rule becoming true by utilizing the definitive finite arrangement of attributes. This work aims to examine and analyze the significant effect of logical permutation on the performance of data extraction ability of the logic mining approach incorporated with the recurrent discrete Hopfield Neural Network. Based on the theory, the effect of permutation and associate memories in recurrent Hopfield Neural Network will potentially improve the accuracy of the existing logic mining approach. To validate the impact of the logical permutation on the retrieval phase of the logic mining model, the proposed work is experimentally tested on a different class of the benchmark real datasets ranging from the multivariate and time-series datasets. The experimental results show the significant improvement in the proposed logical permutation-based logic mining according to the domains such as compatibility, accuracy, and competitiveness as opposed to the plethora of standard 2 Satisfiability Reverse Analysis methods.

Design of single- and multi-loop self-adaptive PID controller using heuristic based recurrent neural network for ALFC of hybrid power system

This paper presents a novel heuristic based recurrent Hopfield neural network (HNN) designed self-adaptive proportional-integral-derivative (PID) controller for automatic load frequency control of interconnected hybrid power system (HPS). The control problem is conceptualized as an optimization problem and solved using a heuristic optimization technique with the aim of minimizing the Lyapunov function. Initially, the energy function is formulated and the differential equations governing the dynamics of HNN are derived. Then, these dynamics are solved using hybrid particle swarm optimization-gravitational search algorithm (PSO-GSA) to obtain the initial solution. The effectiveness of the controller is tested for two-area system considering the system non-linearities and integration of plug-in-electric vehicle (PEV). Further, to improve the speed of response of the system, the cascade control scheme is proposed using the presented approach of heuristic based HNN (h-HNN). The efficacy of the method is examined in single- and multi-loop PID control of three-area HPS. The performance of propounded control schemes is compared with PSO-GSA and generalized HNN based PID controller. The results obtained show that the response of proposed controller is superior in terms of transient and steady state performance indices measured. In addition, the control effort of suggested cascade controller is much reduced compared with other controllers presented. Furthermore, the self-adaptive property of the controller is analyzed for random change in load demand and their corresponding change in gain parameters are recorded. This reveals that the proposed controller is more suitable for stable operation of modern power network with green energy technologies and PEV efficiently.

Computing Generalized Matrix Inverse on Spiking Neural Substrate.

Emerging neural hardware substrates, such as IBM's TrueNorth Neurosynaptic System, can provide an appealing platform for deploying numerical algorithms. For example, a recurrent Hopfield neural network can be used to find the Moore-Penrose generalized inverse of a matrix, thus enabling a broad class of linear optimizations to be solved efficiently, at low energy cost. However, deploying numerical algorithms on hardware platforms that severely limit the range and precision of representation for numeric quantities can be quite challenging. This paper discusses these challenges and proposes a rigorous mathematical framework for reasoning about range and precision on such substrates. The paper derives techniques for normalizing inputs and properly quantizing synaptic weights originating from arbitrary systems of linear equations, so that solvers for those systems can be implemented in a provably correct manner on hardware-constrained neural substrates. The analytical model is empirically validated on the IBM TrueNorth platform, and results show that the guarantees provided by the framework for range and precision hold under experimental conditions. Experiments with optical flow demonstrate the energy benefits of deploying a reduced-precision and energy-efficient generalized matrix inverse engine on the IBM TrueNorth platform, reflecting 10× to 100× improvement over FPGA and ARM core baselines.

The effect of Hebbian plasticity on the attractors of a dynamical system

A central problem in neuroscience is to bridge local synaptic plasticity and the global behavior of a system. It has been shown that Hebbian learning of connections in a feedforward network performs PCA on its inputs [1]. In recurrent Hopfield network with binary units, the Hebbian-learnt patterns form the attractors of the network [2]. Starting from a random recurrent network, Hebbian learning reduces system complexity from chaotic to fixed point [3].

Image restoration using Modified Recurrent Hopfield Neural Network

An approach to restore an image degraded by a blur function and corrupted by random noise is proposed using a Modified Recurrent Hopfield neural network (MRHNN) model. The existing Hopfield network takes more iteration to converge and results in poor quality images as the noise level increases. In the proposed network a fraction of the output of a neuron is fed only to higher order neurons resulting in reduced number of iterations as well as a better SNR. Two updating algorithms: the sequential update; n-simultaneous update are used with the proposed network.

Soft computing in engineering design: a hybrid dual cross-mapping neural network model

Contemporary design process requires the development of a new computational intelligence or soft computing methodology that involves intelligence integration and hybrid intelligent systems for design, analysis and evaluation, and optimization. This paper first presents a discussion of the need to incorporate “intelligence” into an automated design process and the various constraints that designers face when embarking on industrial design projects. Then, it presents the design problem as optimizing the design output against constraints and the use of soft computing and hybrid intelligent systems techniques. In this paper, a soft-computing-integrated intelligent design framework is developed. A hybrid dual cross-mapping neural network (HDCMNN) model is proposed using the hybrid soft computing technique based on “cross-mapping” between a back-propagation network (BPNN) and a recurrent Hopfield network (HNN) for supporting modeling, analysis and evaluation, and optimization tasks in the design process. The two networks perform different but complementary tasks—the BPNN “decides” if the design problem is a “type 0” (rational) or “type 1” (non-rational) problem, and the output layer weights are then used as the energy function for the HNN. The BPNN is used for representing design patterns, training classification boundaries, and outputting network weight values to the HNN, and then the HNN uses the calculated network weight values to evaluate and modify or re-design the design patterns. The developed system provides a unified soft-computing-integrated intelligent design framework with both symbolic and computational intelligence. The system has self-modifying and self-learning functions. Within the system, only one network training is needed for accomplishing the evaluation, rectification/modification, and optimization tasks in the design process. Finally, two case studies are provided to illustrate and validate the developed model and system.

A neural network for shortest path computation

This paper presents a new neural network to solve the shortest path problem for inter-network routing. The proposed solution extends the traditional single-layer recurrent Hopfield architecture introducing a two-layer architecture that automatically guarantees an entire set of constraints held by any valid solution to the shortest path problem. This new method addresses some of the limitations of previous solutions, in particular the lack of reliability in what concerns successful and valid convergence. Experimental results show that an improvement in successful convergence can be achieved in certain classes of graphs. Additionally, computation performance is also improved at the expense of slightly worse results.

Automatic Growing of a Hopfield Style Network During Training for Classification

Hopfield networks, a type of Recurrent Neural Network, may be used as a tool for classification by storing exemplars as memories. This method of using the Hopfield network for classification has certain shortcomings, such as the limits on the number of class exemplars that can be stored and the size of the connection matrix requires when used for classification of images. This paper describes a method of growing a Hopfiled style network for use in classification of patterns. The complete network that is grown consists of three networks in sequence with the middle network being a fully recurrent Hopfield style network. The first network is a one layer feedforward network while the last network is simply a selector network which selects components from the terminal state of the recurrent network. The Hopfield style network grows automatically during training as additional nodes are required. The resulting network can be trained for the purpose of classifying bi-polar vectors. Connection matrices are determined, using a modified Widrow-Hoff learning rule, such that the exemplars are attracted to exemplars or prototypes within the same class. An unclassified element is then classified by the class of its attractor. No pre-processing is required to determine prototypes and all the training elements are used directly. A reduction in the number of arithmetic operations from order of magnitude n2 to n takes place by growing the network rather than initialising the network to size n. The method is successfully applied to the classification of cervical cells for cancer detection and to the classification of diabetes patients recorded in the “Pima Indians Diabetes Data Base”. Results depict how the network grows as learning takes place. © 1997 Elsevier Science Ltd. All Rights Reserved.

Artificial Neural Networks Optimization Method for Radioactive Source Localization

An optimization artificial neural networks model is developed for solving the ill-posed inverse transport problem associated with localizing radioactive sources in a medium with known properties and dimensions. The model is based on the recurrent (or feedback) Hopfield network with fixed weights. The source distribution is determined based on the response of a limited number of external detectors of known spatial deployment in conjunction with a radiation transport model. The algorithm is tested and evaluated for a large number of simulated two-dimensional cases. Computations are carried out at different noise levels to account for statistical errors encountered in engineering applications. The sensitivity to noise is found to depend on the number of detectors and on their spatial deployment. A pretest empirical procedure is, therefore, suggested for determining an effective arrangement of detectors for a given problem.

A RECURRENT NEURAL NETWORK USING TRI-STATE HIDDEN NEURONS TO ORTHOGONALIZE THE MEMORY SPACE

This paper describes a process for adding hidden neurons to a fully recurrent Hopfield neural network in such a way as to optimize the orthogonality of the memory space. The process uses the network itself, operating with a "reverse update rule" to assign optimal values to the hidden neurons for each memory. The outer product rule is used to modify synaptic strengths as each new memory is added. As in a standard Hopfield network this is a fast process because it is noniterative. Tri-state hidden neurons, initially set to zero, are used in the recovery of memories. Simulations indicate that the storage capacity of the network for uncorrelated memories, and the radius of attraction of each memory, are significantly better than those of the standard Hopfield network. The use of hidden neurons permits flexibility in the network capacity for memories of a given length. The network is able to solve second-order hetero-associative problems, as illustrated with solutions to the XOR set of associations.