Hopfield Neural Network Research Articles

Cocaine diminishes functional network robustness and destabilizes the energy landscape of neuronal activity in the medial prefrontal cortex.

We present analysis of neuronal activity recordings from a subset of neurons in the medial prefrontal cortex of rats before and after the administration of cocaine. Using an underlying modern Hopfield model as a description for the neuronal network, combined with a machine learning approach, we compute the underlying functional connectivity of the neuronal network. We find that the functional connectivity changes after the administration of cocaine with both functional-excitatory and functional-inhibitory neurons being affected. Using conventional network analysis, we find that the diameter of the graph, or the shortest length between the two most distant nodes, increases with cocaine, suggesting that the neuronal network is less robust. We also find that the betweenness centrality scores for several of the functional-excitatory and functional-inhibitory neurons decrease significantly, while other scores remain essentially unchanged, to also suggest that the neuronal network is less robust. Finally, we study the distribution of neuronal activity and relate it to energy to find that cocaine drives the neuronal network towards destabilization in the energy landscape of neuronal activation. While this destabilization is presumably temporary given one administration of cocaine, perhaps this initial destabilization indicates a transition towards a new stable state with repeated cocaine administration. However, such analyses are useful more generally to understand how neuronal networks respond to perturbations.

Impacts of Resampling and Downscaling Digital Elevation Model and Its Morphometric Factors: A Comparison of Hopfield Neural Network, Bilinear, Bicubic, and Kriging Interpolations

The digital elevation model (DEM) and its derived morphometric factors, i.e., slope, aspect, profile and plan curvatures, and topographic wetness index (TWI), are essential for natural hazard modeling and prediction as they provide critical information about the terrain’s characteristics that can influence the likelihood and severity of natural hazards. Therefore, increasing the accuracy of the DEM and its derived factors plays a critical role. The primary aim of this study is to evaluate and compare the effects of resampling and downscaling the DEM from low to medium resolution and from medium to high resolutions using four methods: namely the Hopfield Neural Network (HNN), Bilinear, Bicubic, and Kriging, on five morphometric factors derived from it. A geospatial database was established, comprising five DEMs with different resolutions: specifically, a SRTM DEM with 30 m resolution, a 20 m resolution DEM derived from topographic maps at a scale of 50,000, a 10 m resolution DEM generated from topographic maps at a scale of 10,000, a 5 m resolution DEM created using surveying points with total stations, and a 5 m resolution DEM constructed through drone photogrammetry. The accuracy of the resampling and downscaling was assessed using Root Mean Square Error (RMSE) and mean absolute error (MAE) as statistical metrics. The results indicate that, in the case of downscaling from low to medium resolution, all four methods—HNN, Bilinear, Bicubic, and Kriging—significantly improve the accuracy of slope, aspect, profile and plan curvatures, and TWI. However, for the case of medium to high resolutions, further investigations are needed as the improvement in accuracy observed in the DEMs does not necessarily translate to the improvement of the second derivative morphometric factors such as plan and profile curvatures and TWI. While RMSEs of the first derivatives of DEMs, such as slope and aspect, reduced in a range of 8% to 55% in all five datasets, the RMSEs of curvatures and TWI slightly increased in cases of downscaling and resampling of Dataset 4. Among the four methods, the HNN method provides the highest accuracy, followed by the bicubic method. The statistics showed that in all five cases of the experiment, the HNN downscaling reduced the RMSE and MAE by 55% for the best case and 10% for the worst case for slope, and it reduced the RMSE by 50% for the best case of aspect. Both the HNN and the bicubic methods outperform the Kriging and bilinear methods. Therefore, we highly recommend using the HNN method for downscaling DEMs to produce more accurate morphometric factors, slope, aspect, profile and plan curvatures, and TWI.

Refined troposphere delay models by NWM ray-tracing for pseudolite positioning system and their performance assessment

Pseudolite positioning system can enhance GNSS (Global Navigation Satellite System) by improving satellite geometry and providing services independently in the GNSS unavailable environment. Many error items of pseudolite positioning system, e.g., tropospheric delay error, need to be corrected by an accurate model. While several tropospheric delay models suitable for pseudolite positioning systems have been proposed, the key parameters indicating the upper boundaries for tropospheric refraction are fixed to an experimental value in these models. In this paper, we studied the spatiotemporal characteristics of the upper boundary heights for hydrostatic and wet tropospheric delay based on the ERA5 global meteorological reanalysis data. With the refined height boundary term, we refine four existing tropospheric delay models for pseudolite positioning systems, including RTCA, MRTCA, Bouska and Hopfield models. To evaluate the performance of the refined models, we selected 12 airports distributed globally as experimental areas and analyzed the error characteristics of the four refined tropospheric models and the LTC model (independent of height). The experimental results show that when the slant distance between the pseudolite base station and the receiver is constant at 5 km, the MRTCA model has the best performance, with an RMSE (root mean square error) of about 0.15 m. When the elevation angle between the pseudolite base station and the receiver is constant, the RMSE of the LTC model is the smallest at the low elevation angle, and the RMSE of the MRTCA model is the smallest at the high elevation. The RMSE of pseudolite tropospheric delay models show seasonal variation.

Exponential Capacity of Dense Associative Memories.

Recent generalizations of the Hopfield model of associative memories are able to store a number P of random patterns that grows exponentially with the number N of neurons, P=exp(αN). Besides the huge storage capacity, another interesting feature of these networks is their connection to the attention mechanism which is part of the Transformer architecture widely applied in deep learning. In this work, we study a generic family of pattern ensembles using a statistical mechanics analysis which gives exact asymptotic thresholds for the retrieval of a typical pattern, α_{1}, and lower bounds for the maximum of the load α for which all patterns can be retrieved, α_{c}, as well as sizes of attraction basins. We discuss in detail the cases of Gaussian and spherical patterns, and show that they display rich and qualitatively different phase diagrams.

Hebbian dreaming for small datasets

The dreaming Hopfield model constitutes a generalization of the Hebbian paradigm for neural networks, that is able to perform on-line learning when “awake” and also to account for off-line “sleeping” mechanisms. The latter have been shown to enhance storing in such a way that, in the long sleep-time limit, this model can reach the maximal storage capacity achievable by networks equipped with symmetric pairwise interactions. In this paper, we inspect the minimal amount of information that must be supplied to such a network to guarantee a successful generalization, and we test it both on random synthetic and on standard structured datasets (i.e., MNIST, Fashion-MNIST and Olivetti). By comparing these minimal thresholds of information with those required by the standard (i.e., always “awake”) Hopfield model, we prove that the present network can save up to ∼90% of the dataset size, yet preserving the same performance of the standard counterpart. This suggests that sleep may play a pivotal role in explaining the gap between the large volumes of data required to train artificial neural networks and the relatively small volumes needed by their biological counterparts. Further, we prove that the model Cost function (typically used in statistical mechanics) admits a representation in terms of a standard Loss function (typically used in machine learning) and this allows us to analyze its emergent computational skills both theoretically and computationally: a quantitative picture of its capabilities as a function of its control parameters is achieved and consistency between the two approaches is highlighted.The resulting network is an associative memory for pattern recognition tasks that learns from examples on-line, generalizes correctly (in suitable regions of its control parameters) and optimizes its storage capacity by off-line sleeping: such a reduction of the training cost can be inspiring toward sustainable AI and in situations where data are relatively sparse.

Memristor-cascaded hopfield neural network with attractor scroll growth and STM32 hardware experiment

This paper proposes a memristor-cascaded Hopfield neural network (MC-HNN), which is constructed by directly cascading two bi-neuron Hopfield neural networks (HNNs) using a memristor synapse. With its mathematical model, the stability of line equilibrium set is analyzed, and two representative chaotic attractors are revealed. On this basis, the multi-scroll chaotic attractor (MSCA) with scroll growth over time is investigated. The results manifest that MC-HNN can not only generate MSCA with unidirectional and bidirectional scroll-growths, but also produce initial-offset coexisting attractors. In particular, the generated MSCA only involves the line equilibrium set, so Shil'nikov's theorem does not apply in this case. Finally, STM32 hardware board is developed, and the numerically simulated results are captured experimentally by oscilloscope.

Neural network-based modeling of solid oxide fuel cells for marine applications

As the solid oxide fuel cell (SOFC) experimental test is still quite cost-effective and time-consuming, there is a growing need for developing effective simulation tools to reduce the time and cost of the marine SOFC performance test and optimization. The present paper is aimed to study the modeling and simulation of marine solid oxide fuel cells by artificial intelligence method. A neural network based on a particle swarm optimization algorithm is used to establish a marine solid oxide fuel cell model for voltage/current characteristic analysis. The model is also compared with BP neural network and Hopfield neural network methods. The simulation results compared with experimental data show that the effectivity of the particle swarm optimization neural network algorithm is best, which can accurately predict the voltage/current characteristic curves of a SOFC under different fuel flow-air volume ratios. The model study can provide support for SOFC performance characteristics analyses and has significant potential in SOFC optimization applications.

Stochastic Antiresonance for Systems with Multiplicative Noise and Sector-Type Nonlinearities.

The paradigm of stochastic antiresonance is considered for a class of nonlinear systems with sector bounded nonlinearities. Such systems arise in a variety of situations such as in engineering applications, in physics, in biology, and in systems with more general nonlinearities, approximated by a wide neural network of a single hidden layer, such as the error equation of Hopfield networks with respect to equilibria or visuo-motor tasks. It is shown that driving such systems with a certain amount of state-multiplicative noise, one can stabilize noise-free unstable systems. Linear-Matrix-Inequality-based stabilization conditions are derived, utilizing a novel non-quadratic Lyapunov functional and a numerical example where state-multiplicative noise stabilizes a nonlinear system exhibiting chaotic behavior is demonstrated.

Noise tailoring, noise annealing, and external perturbation injection strategies in memristive Hopfield neural networks

The commercial introduction of a novel electronic device is often preceded by a lengthy material optimization phase devoted to the suppression of device noise as much as possible. The emergence of novel computing architectures, however, triggers a paradigm shift in noise engineering, demonstrating that non-suppressed but properly tailored noise can be harvested as a computational resource in probabilistic computing schemes. Such a strategy was recently realized on the hardware level in memristive Hopfield neural networks, delivering fast and highly energy efficient optimization performance. Inspired by these achievements, we perform a thorough analysis of simulated memristive Hopfield neural networks relying on realistic noise characteristics acquired on various memristive devices. These characteristics highlight the possibility of orders of magnitude variations in the noise level depending on the material choice as well as on the resistance state (and the corresponding active region volume) of the devices. Our simulations separate the effects of various device non-idealities on the operation of the Hopfield neural network by investigating the role of the programming accuracy as well as the noise-type and noise amplitude of the ON and OFF states. Relying on these results, we propose optimized noise tailoring and noise annealing strategies, comparing the impact of internal noise to the effect of external perturbation injection schemes.

Development of a hybrid intelligent switching hydraulically interconnected suspension system under a multi-objective optimized mode selection strategy with real-world condition

Reduction in vertical acceleration is crucial for car manufacturers. This parameter evaluates the comfort index. Moreover, the handling index is another feature that must be under investigation. The suspension system has an essential impact on these two indices. A hybrid intelligent switching hydraulically interconnected suspension (HISHIS) is proposed. The parallel configuration is combined with the diagonal configuration in the suspension system. A mode selection strategy is discussed to select the layout between diagonal and parallel configurations. The only mode of parallel configuration is called Anti-pitch configuration. The diagonal configuration has three modes, including Anti-roll, Anti-oversteering, and Anti-vibration configuration. According to the mode selection strategy, one of these three diagonal configuration modes might be selected. The origin of roll and pitch generation is inertia force and road inputs. Hopfield neural network can recognize the origin of roll and pitch generation. Therefore, the performance of valves will change according to Hopfield neural network recognition. The results of different maneuvers show the improvement at each targeted parameter in various tasks independently. The data logger gathered the acceleration of the vehicle in real-world conditions. However, the margins of the selection strategy block are multi-objective optimized with a genetic algorithm to reach better responses in real-world conditions. The roll angle, yaw rate, allowable exposure time, and pitch angle are improved by 63%, 5%, 40%, and 99% on average. Also, optimizing the selection strategy improves the allowable exposure time by 9%. Obviously, by combining two layouts, it is possible to have a flexible situation to improve ride comfort and handling situations. In addition, there is a conditional strategy to select different layouts and modes to reach a better response.

SOFC stack modeling: a hybrid RBF-ANN and flexible Al-Biruni Earth radius optimization approach

Abstract This study introduces a novel hybrid methodology for model identification of solid oxide fuel cell (SOFC) stacks by integrating a radial basis function-based artificial neural network (RBF-ANN) with a flexible Al-Biruni Earth radius optimizer (FA-BERO). The primary objective of the proposed method is to augment the precision and efficiency of SOFC stack modeling by considering the advantages of both RBF-ANN and FA-BERO algorithms. The main purpose of using these two methods is to optimize the structure of the RBF-ANN based on the suggested FA-BERO algorithm. The other contribution of this study is improving the efficiency of the Al-Biruni Earth radius optimizer (A-BERO) by applying two improvements on it, including constriction factor and elimination phase to increase the exploration and exploitation strength of the basic A-BERO. To validate the effectiveness of the proposed model, it is compared with some state-of-the-art models in the field, such as the artificial neural network and multi-armed bandit algorithm (ANN/MABA) and rotor Hopfield neural network and grey wolf optimization (RHNN/GWO). Furthermore, the model is validated by experimental data, and the final results demonstrate the efficacy of the hybrid approach in accurately representing the intricate behavior of SOFC stacks. The proposed model achieves lower error rates (ERs) and root mean squared errors (RMSEs) than the comparative methods across different network arrangements and temperature conditions. The results show that, for instance, for the 2/12/1 network arrangement at 900°C, the proposed model attains an ER of 6.69% and an RMSE of 2.13, while the ANN/MABA and RHNN/GWO methods obtain ERs of 9.67% and 8.54%, as well as RMSE values of 24.48 and 9.23, respectively. The proposed model also exhibits superior accuracy and convergence speed compared to the comparative methods, as shown by the current–voltage curves and the convergence analysis. Consequently, this novel hybrid methodology offers a valuable tool for researchers and engineers working in the domain of fuel cell technology, enabling them to better understand and optimize SOFC stack performance.

Bionic modeling and dynamics analysis of heterogeneous brain regions connected by memristive synaptic crosstalk

The coupling dynamics between different brain regions has long been a gap in the study of neural dynamics. To overcome this bottleneck, the synaptic crosstalk characterized by coupled memristors is creatively connected to different brain regions characterized by two heterogeneous Hopfield neural networks (HNNs), forming a novel bionic memristive synaptic crosstalk coupled-HNN (MSC-HNN) model for the first time. Rigorous boundedness and stability analyses ensure that coupling dynamics are generated within a bounded region. Utilizing innovative dynamics methods, it is revealed that the plasticity of synaptic crosstalk allows MSC-HNN to exhibit homogeneous dynamics properties globally, and the two internal HNNs preserve heterogeneous bifurcation processes and attractors. Interestingly, when the initial conditions undergo slight variations, the multistability phenomenon is simultaneously represented by the coexisting parameter bifurcation and the local basin of attraction, which enhances the diversity of activities in MSC-HNN. Furthermore, these novel dynamics effects are evaluated and validated through quantitative calculations of complexity measures and spectra. Subsequently, the developed analog and digital circuit implementation paradigms confirm the feasibility and practical value of MSC-HNN.

Dynamics analysis, synchronization and FPGA implementation of multiscroll Hopfield neural networks with non-polynomial memristor

The number of attractors in a memristor-based multiscroll Hopfield Neural Network (HNN) is typically coupled with the number of polynomials, which leads to a coupling between the computational complexity and resource utilization in circuit implementation. To decouple this relationship, we propose a non-polynomial memristor that satisfies the Lipschitz condition. Regardless of whether it is used to simulate synaptic behavior, simulate the impact of electromagnetic radiation, or a combination of both scenarios, it can conveniently control the generation of single-direction or multiple-direction multiscroll attractors without adding or reducing any terms. By constructing Lyapunov functions, the sufficient condition for these multiscroll memristor HNNs to be bounded is obtained. After improving the feasibility of linear matrix inequalities, a strongly adaptive observer is proposed. After uniting an adaptive sliding mode control method, we propose a new adaptive synchronization scheme to simulate neural network synchronization. Finally, the digital circuit implementation and functional verification of these memristor-based multiscroll HNNs are completed using a field-programmable gate array (FPGA). Based on this, an image encryption circuit is designed so that the FPGA can directly encrypt images and transmit them to the IO device.

Dynamics of a random Hopfield neural lattice model with adaptive synapses and delayed Hebbian learning

UDC 517.9 A Dong–Hopfield neural lattice model with random external forcing and delayed response to the evolution of interconnection weights is developed and studied. The interconnection weights evolve according to the Hebbian learning rule with a decay term and contribute to changes in the states after a short delay. The lattice system is first reformulated as a coupled functional-ordinary differential equation system on an appropriate product space. Then the solution of the system is shown to exist and be unique. Furthermore it is shown that the system of equations generates a continuous random dynamical system. Finally, the existence of random attractors for the random dynamical system generated by the Dong–Hopfield model is established.

Logic mining with hybridized 3-satisfiability fuzzy logic and harmony search algorithm in Hopfield neural network for Covid-19 death cases

<abstract> <p>Since the beginning of the Covid-19 infections in December 2019, the virus has emerged as the most lethally contagious in world history. In this study, the Hopfield neural network and logic mining technique merged to extract data from a model to provide insight into the link between factors influencing the Covid-19 datasets. The suggested technique uses a 3-satisfiability-based reverse analysis (3SATRA) and a hybridized Hopfield neural network to identify the relationships relating to the variables in a set of Covid-19 data. The list of data is to identify the relationships between the key characteristics that lead to a more prolonged time of death of the patients. The learning phase of the hybridized 3-satisfiability (3SAT) Hopfield neural network and the reverse analysis (RA) method has been optimized using a new method of fuzzy logic and two metaheuristic algorithms: Genetic and harmony search algorithms. The performance assessment metrics, such as energy analysis, error analysis, computational time, and accuracy, were computed at the end of the algorithms. The multiple performance metrics demonstrated that the 3SATRA with the fuzzy logic metaheuristic algorithm model outperforms other logic mining models. Furthermore, the experimental findings have demonstrated that the best-induced logic identifies important variables to detect critical patients that need more attention. In conclusion, the results validate the efficiency of the suggested approach, which occurs from the fact that the new version has a positive effect.</p> </abstract>

Exploration of Energy Meter Data Security and Fast Response Technology in Spot Power Trading

Abstract With the advancement of smart grid construction, higher requirements have been put forward for energy meter data, which need to realize real-time, accurate, efficient, safe, and economical data transmission. In this paper, an intelligent collection of energy meter data is designed to monitor energy meter data. The PCA algorithm is utilized to downscale the energy meter data, and the LMR algorithm is combined to monitor abnormal data from the meter. Based on the smart contract in blockchain technology, a parallel Pedersen commitment algorithm based on privacy protection is designed, a hybrid signature algorithm is created to ensure the secure transmission of energy meter data and a cluster load balancing model for energy meter data is also designed in combination with Hopfield neural network. The PCA-LMR algorithm identifies 95 anomalies of the data in 12 months and under the encryption of a smart contract. The packet loss rate of energy meter data is maintained at about 2%, and after optimizing the cluster load capacity of energy meter data using the Hopfield neural network, the cluster load variance value is about 1.84 in 20 h. Modern technology can improve the secure transmission of energy meter data and enhance its rapid response ability to ensure the economic benefits of the power system.

A novel compound exponential locally active memristor coupled Hopfield neural network

The neural network model coupled with memristors has been extensively studied due to its ability to more accurately represent the complex dynamic characteristics of the biological nervous system. Currently, the mathematical model of memristor used to couple neural networks mainly focuses on primary function, absolute value function, hyperbolic tangent function, etc. To further enrich the memristor-coupled neural network model and take into account the motion law of particles in some doped semiconductors, a new compound exponential local active memristor is proposed and used as a coupling synapse in the Hopfield neural network. Using the basic dynamic analysis method, the system’s dynamic behaviors are studied under different parameters and the coexistence of multiple bifurcation modes under different initial values. In addition, the influence of frequency change of external stimulation current on the system is also studied. The experimental results show that the internal parameters of memristor synapses regulate the system, and the system has a rich dynamic behavior, including symmetric attractor coexistence, asymmetric attractor coexistence, large-scale chaos as shown in attached figure, and bursting oscillation. Finally, the hardware of the system is realized by the STM32 microcontroller, and the experimental results verify the realization of the system.

Reliability assessment and optimization of computer networks based on neural networks

Abstract Amidst the swift evolution of computer technologies, the prevalence of computer networks has become pivotal across diverse sectors, increasingly reliant on their robust functionality. This study rigorously evaluates and enhances the reliability of computer networks by leveraging an index system and an evaluation model devised through neural networks. To strengthen network dependability, this research employs a Hopfield neural network to address multi-constraint Quality of Service (QoS) multicast routing challenges, thereby elevating network reliability. Simulation experiments demonstrate that the Hopfield neural network effectively mitigates network latency and exhibits superior convergence performance compared to conventional QoS multicast routing methods. Further, this paper applies the neural network-based evaluation model to analyze the reliability of the air traffic network within the H-area after integrating the optimized computer network framework. It is observed that the most significant contributor to traffic flow loss is the network's degree value. An analysis of traffic flow density, employing actual sector flow data, reveals that high traffic volumes typically precipitate congestion. Nonetheless, the traffic flow density value consistently exceeds 100, suggesting that the enhanced computer network model holds practical applicability in real-world scenarios.

A novel processor for dynamic evolution of constrained SAT problems: The dynamic evolution variant of the discrete Hopfield neural network satisfiability model

The Boolean satisfiability problem, a renowned NP-complete challenge in computer science, has recently garnered interest in the Discrete Hopfield Neural Network - Satisfiability model. This model adeptly integrates logical rules into Hopfield networks, excelling in locating global minima for traditional SAT problems. However, it faces efficiency challenges when dealing with SAT problems characterized by dynamic evolution constraints due to its static network architecture. During dynamic evolution iterations, there is a significant exponential increase in the computational costs due to redundant and repetitive computations.In order to address this challenge, this paper introduces a dynamic evolution variant of the Discrete Hopfield Neural Network - Satisfiability model. In extensive simulation experiments, we progressively augmented the number of constraint clauses from 1 to 1500, seeking global minima for CNF problems. The proposed model exhibited congruent performance with the traditional model, achieving a Global Minimum Ratio of 1 and a Hamming distance of 0. Crucially, the proposed model minimized CPU utilization and neared zero error metrics, while the traditional model experienced exponential CPU and error metric escalation. These outcomes affirm the proposed model's robust global search capabilities and high precision, aligning with the traditional model. Furthermore, owing to this model's incorporation of not only temporal constraint increment operator but also innovative real-time learning techniques and clever integration methods, along with the establishment of a novel real-time decision mechanism, the proposed model effectively addresses the issues of redundancy and repeated calculations inherent in traditional models. This results in a stable and significantly improved computational speed. Additionally, this model's dynamic evolution network architecture is specifically designed to accommodate arbitrary and efficient extensions of dynamic constraints, while also featuring a built-in preprocessing function to filter out duplicate variables. In the future, as a dynamic evolution processor, this model exhibits immense potential for solving large-scale SAT problems with dynamic evolution constraints and holds broad application prospects in emerging fields such as automated reasoning, intelligent recommendation systems, and machine learning.

Outperforming RBM Feature-Extraction Capabilities by "Dreaming" Mechanism.

Inspired by a formal equivalence between the Hopfield model and restricted Boltzmann machines (RBMs), we design a Boltzmann machine, referred to as the dreaming Boltzmann machine (DBM), which achieves better performances than the standard one. The novelty in our model lies in a precise prescription for intralayer connections among hidden neurons whose strengths depend on features correlations. We analyze learning and retrieving capabilities in DBMs, both theoretically and numerically, and compare them to the RBM reference. We find that, in a supervised scenario, the former significantly outperforms the latter. Furthermore, in the unsupervised case, the DBM achieves better performances both in features extraction and representation learning, especially when the network is properly pretrained. Finally, we compare both models in simple classification tasks and find that the DBM again outperforms the RBM reference.