Continuous Hopfield Network Research Articles

OPT-FRAC-CHN: Optimal Fractional Continuous Hopfield Network

The continuous Hopfield network (CHN) is a common recurrent neural network. The CHN tool can be used to solve a number of ranking and optimization problems, where the equilibrium states of the ordinary differential equation (ODE) related to the CHN give the solution to any given problem. Because of the non-local characteristic of the “infinite memory” effect, fractional-order (FO) systems have been proved to describe more accurately the behavior of real dynamical systems, compared to the model’s ODE. In this paper, a fractional-order variant of a Hopfield neural network is introduced to solve a Quadratic Knap Sac Problem (QKSP), namely the fractional CHN (FRAC-CHN). Firstly, the system is integrated with the quadratic method for fractional-order equations whose trajectories have shown erratic paths and jumps to other basin attractions. To avoid these drawbacks, a new algorithm for obtaining an equilibrium point for a CHN is introduced in this paper, namely the optimal fractional CHN (OPT-FRAC-CHN). This is a variable time-step method that converges to a good local minima in just a few iterations. Compared with the non-variable time-stepping CHN method, the optimal time-stepping CHN method (OPT-CHN) and the FRAC-CHN method, the OPT-FRAC-CHN method, produce the best local minima for random CHN instances and for the optimal feeding problem.

Parameter tuning of continuous Hopfield network applied to combinatorial optimization

Parameter tuning of continuous Hopfield network applied to combinatorial optimization

Improving Continuous Hopfield Network Stability Using Runge-Kutta Method

Continuous Hopfield network is a recurring network that has shown its ability to solve important optimization problems. Continuous Hopfield network dynamic system is characterized by a differential equation. This equation is difficult to solve, especially for large problems. This led researchers to discretize the differential equation using Euler’s method. However, this method generally does not converge to a good solution because it is sensitive to the step size decision and initial conditions. In this work, we discretize the dynamic system of continuous Hopfield network by a new method of Runge-Kutta. This method is strong in terms of stability and performance in order to converge to a better solution. This new method introduces two phases for better network stability. The first phase targets to solve the dynamic equation by the Euler method, while the second phase allows refining the solution found in the first phase. Experimental results on benchmarks show that the proposed approach can effectively improve Hopfield neural network performance.

Clustering Based on Continuous Hopfield Network

Clustering aims to group n data samples into k clusters. In this paper, we reformulate the clustering problem into an integer optimization problem and propose a recurrent neural network with n×k neurons to solve it. We prove the stability and convergence of the proposed recurrent neural network theoretically. Moreover, clustering experiments demonstrate that the proposed clustering algorithm based on the recurrent neural network can achieve the better clustering performance than existing clustering algorithms.

Max stable set problem to found the initial centroids in clustering problem

In this paper, we propose a new approach to solve the document-clustering using the K-Means algorithm. The latter is sensitive to the random selection of the k cluster centroids in the initialization phase. To evaluate the quality of K-Means clustering we propose to model the text document clustering problem as the max stable set problem (MSSP) and use continuous Hopfield network to solve the MSSP problem to have initial centroids. The idea is inspired by the fact that MSSP and clustering share the same principle, MSSP consists to find the largest set of nodes completely disconnected in a graph, and in clustering, all objects are divided into disjoint clusters. Simulation results demonstrate that the proposed K-Means improved by MSSP (KM_MSSP) is efficient of large data sets, is much optimized in terms of time, and provides better quality of clustering than other methods.

Quadratic Convex Reformulation for Solving Task Assignment Problem with Continuous Hopfield Network

This research is an optimal allocation of tasks to processors in order to minimize the total costs of execution and communication. This problem is called the Task Assignment Problem (TAP) with nonuniform communication costs. To solve the latter, the first step concerns the formulation of the problem by an equivalent zero-one quadratic program with a convex objective function using a convexification technique, based on the smallest eigenvalue. The second step concerns the application of the Continuous Hopfield Network (CHN) to solve the obtained problem. The calculation results are presented for the instances from the literature, compared to solutions obtained both the CPLEX solver and by the heuristic genetic algorithm, and show an improvement in the results obtained by applying only the CHN algorithm. We can see that the proposed approach evaluates the efficiency of the theoretical results and achieves the optimal solutions in a short calculation time.

QoS routing in cluster OLSR by using the artificial intelligence model MSSP in the big data environnment

<p><span id="docs-internal-guid-75ba661c-7fff-7cfb-9afe-b7c901c3fe82"><span>The most complex problems, in data science and more specifically in artificial intelligence, can be modeled as cases of the maximum stable set problem (MSSP). this article describes a new approach to solve the MSSP problem by proposing the continuous hopfield network (CHN) to build optimized link state protocol routing (OLSR) protocol cluster. our approach consists in proposing in two stages: the first acts at the level of the choice of the OLSR master cluster in order to quickly make a local minimum using the CHN, by modeling the MSSP problem. As for the second step, the objective is the improvement of the precision making a solution of efficient at the first rank of neighborhood as a linear constraint, and at the end, to find the resolution of the model using the CHN. We will show that this model determines a good solution of the MSSP problem. To test the theoretical results, we propose a comparison with a classic OLSR.</span></span></p>

A New Hybrid Routing Protocol Using a Modified K-Means Clustering Algorithm and Continuous Hopfield Network for VANET

Vehicular Ad-hoc Networks (VANET) offer several user applications for passengers and drivers, as well as security and internet access applications. To ensure efficient data transmission between vehicles, a reliable routing protocol is considered a significant challenge. This paper suggests a new clustering-based routing protocol combining a modified K-Means algorithm with Continuous Hopfield Network and Maximum Stable Set Problem (KMRP) for VANET. In this way, the basic input parameters of the K-Means algorithm, such as the number of clusters and the initial cluster heads, will not be selected randomly, but using Maximum Stable Set Problem and Continuous Hopfield Network. Then the assignment of vehicles to clusters will be carried out according to Link Reliability Model as a metric that replaces the distance parameter in the K-Means algorithm. Finally, the cluster head is selected by weight function according to the amount of free buffer space, the speed, and the node degree. The simulation results have proved that the designed protocol performs better in a highway vehicular environment, compared to the most recent schemes designed for the same objective. In fact, KMRP reduces traffic congestion, and thus provides a significant increase in Throughput. In addition, KMRP decreases the transmission delay and guarantees the stability of the clusters in high density and mobility, which acts better in terms of the Packet Delivery Ratio.

The 2-opt behavior of the Hopfield Network applied to the TSP

The Continuous Hopfield Network (CHN) became one of the major breakthroughs in the come back of Neural Networks in the mid 80s, as it could be used to solve combinatorial optimization problems such as the Traveling Salesman Problem. Once researchers provided a mechanism, not based in trial-and-error, to guarantee the feasibility of the CHN, the quality of the solution was inferior to the ones provided by other heuristics. The next natural step is to study the behavior of the CHN as an optimizer, in order to improve its performance. With this regard, this paper analyzes the attractor basins of the CHN and establishes the mathematical foundations that guarantee the behavior of the network as a 2-opt; with the aim to open a new research line in which the CHN may be used, given the appropriate parameter setting, to solve a k-opt, which would make the network highly competitive. The analysis of the attraction basins of the CHN and its interpretation as a 2-opt is the subject of this article.

Determining the Number of Clusters using Neural Network and Max Stable Set Problem

One of the most difficult problems, in cluster analysis is the determination of the number of clusters in a data set. Solving this problem consists in detecting and finding the best number of clusters, which is an input parameter for the clustering problems. In this paper, we propose a new approach using the Maximum Stable Set Problem (MSSP) combined by Continuous Hopfield Network (CHN) to determine the number of clusters, which is a basic input parameter for K-Means method. By testing the theoretical results, the proposed approach was validated on a real application for the text mining. Some numerical examples and computational experiments assess the effectiveness of this approach as demonstrated in this paper.

HopLand: single-cell pseudotime recovery using continuous Hopfield network-based modeling of Waddington’s epigenetic landscape

MotivationThe interpretation of transcriptional dynamics in single-cell data, especially pseudotime estimation, could help understand the transition of gene expression profiles. The recovery of pseudotime increases the temporal resolution of single-cell transcriptional data, but is challenging due to the high variability in gene expression between individual cells. Here, we introduce HopLand, a pseudotime recovery method using continuous Hopfield network to map cells to a Waddington’s epigenetic landscape. It reveals from the single-cell data the combinatorial regulatory interactions among genes that control the dynamic progression through successive cell states. ResultsWe applied HopLand to different types of single-cell transcriptomic data. It achieved high accuracies of pseudotime prediction compared with existing methods. Moreover, a kinetic model can be extracted from each dataset. Through the analysis of such a model, we identified key genes and regulatory interactions driving the transition of cell states. Therefore, our method has the potential to generate fundamental insights into cell fate regulation.Availability and implementationThe MATLAB implementation of HopLand is available at https://github.com/NetLand-NTU/HopLand.

Improving the Hopfield model performance when applied to the traveling salesman problem

The continuous Hopfield network (CHN) can be used to solve, among other combinatorial optimization problems, the traveling salesman problem (TSP). In order to improve the performance of this heuristic technique, a divide-and-conquer strategy based on two phases is proposed. The first phase involves linking cities with the most neighbors to define a set of chains of cities and, secondly, to join these with isolated cities to define the final tour. Both problems are solved by mapping the two TSPs onto their respective CHNs. The associated parameter-setting procedures are deduced; these procedures ensure the feasibility of the obtained tours, and the quality of the solutions is compared with the pure CHN approach using some traveling salesman problem library (TSPLIB) instances. By means of this strategy, solving TSP instances with sizes of up to 13,509 cities are allowed with the computational resources we had available. Finally, the new divide-and-conquer procedure is improved by tuning the parameter which controls the first phase.

Solving the Weighted Constraint Satisfaction Problems Via the Neural Network Approach

A wide variety of real world optimization problems can be modelled as Weighted Constraint Satisfaction Problems (WCSPs). In this paper, we model this problem in terms of in original 0-1 quadratic programming subject to leaner constraints. View it performance, we use the continuous Hopfield network to solve the obtained model basing on original energy function. To validate our model, we solve several instance of benchmarking WCSP. In this regard, our approach recognizes the optimal solution of the said instances.

A discrete gradient method to enhance the numerical behaviour of Hopfield networks

This paper presents the construction of a numerical method for implementing algorithms that are based on a gradient flow. In particular, continuous Hopfield networks for solving optimization problems are considered as a case in point. The focus is the preservation of the favourable properties of the continuous system under discretization. Firstly, the conventional discretization is formulated as a non-standard numerical method that solves the continuous equation, depending on the step size. A rigourous theoretical analysis shows that it is a consistent method, but it fails to preserve the gradient nature, since periodic solutions occur, so no Lyapunov function can exist. Next we present the construction of a method that preserves the Lyapunov function of the original differential equation regardless of the step size. This procedure, based upon discrete gradients, yields an algorithm that preserves the Lyapunov function of the continuous system, thus reproducing the same qualitative behaviour, namely stability of equilibria and convergence to solutions. A remarkable property of the proposed technique is that it can be computed explicitly as long as the Lyapunov function is multi-linear, which is the case of Hopfield neural networks. This results in enhanced performance, compared to the conventional discretization, as shown in a comprehensive set of numerical experiments.

Neural networks using two-component Bose-Einstein condensates

The authors previously considered a method of solving optimization problems by using a system of interconnected network of two component Bose-Einstein condensates (Byrnes, Yan, Yamamoto New J. Phys. 13, 113025 (2011)). The use of bosonic particles gives a reduced time proportional to the number of bosons N for solving Ising model Hamiltonians by taking advantage of enhanced bosonic cooling rates. Here we consider the same system in terms of neural networks. We find that up to the accelerated cooling of the bosons the previously proposed system is equivalent to a stochastic continuous Hopfield network. This makes it clear that the BEC network is a physical realization of a simulated annealing algorithm, with an additional speedup due to bosonic enhancement. We discuss the BEC network in terms of neural network tasks such as learning and pattern recognition and find that the latter process may be accelerated by a factor of N.

Multistability in large scale models of brain activity

Rich spatiotemporal dynamical patterns, observed in the brain at rest, reveal several large-scale functional networks, presumably involved in different brain functions. In parallel, structural networks obtained by Diffusion Spectrum Imaging (connectome) identify several interconnected sub-networks that overlap with the functional networks [3]. Neural mass model simulations aim at developing realistic models of the brain activity. In particular, multiple attractors can be identified and spontaneous alternation between several brain states can be obtained [2]. The number and the organization of the fixed-point attractors strongly vary depending of the parameters used in the model. In order to provide a general view of connectome-based dynamical systems, we developed a simplified neural mass model inspired by the continuous Hopfield network [4], to which we add a stochastic component, and a dynamic threshold (eq.2) that prevents reaching the two trivial attractors of system (eq.1). τdxidt=-xi+∑j=1NWi,jφxj,γ,θ+σ2ηi(t) (1) τθdθdt=-θ+ΩN∑i=1Nφxi,γ,θ (2) φxi,γ,θ=121+tanhγxi-θ (3) where W is the N = 66 nodes human connectome [3], x is the node potential, φ is the node output, θ is the adaptive threshold, η(t) is a white noise and σ the diffusion parameter. Three main parameters (excitation strength ||W||, inhibition strength Ω and node excitability γ) are systematically varied in order to identify the fixed points attractors of the dynamics (Figure ​(Figure11). Figure 1 Number of attractors obtained for different (Ω, ||W||) couples with γ = 30. Attractor patterns isodensity lines are indicated in black The adaptive threshold plays the role of a global inhibition and allows stabilizing many attractors, where the Ω / ||W|| ratio controls the sparsity of the activity. Then multistable dynamical systems are obtained in a large region of the parameter space, allowing to identify many different attractor patterns. The number of attractors is found to increase with the value of gamma (node excitability), at the expense of their intrinsic stability. Itinerant dynamics is obtained when a significant noise level is introduced in the system. Then, several attractive patterns can be reached on a single trajectory, where the duration of the visit reflects the stability of the pattern. The simple dynamical system we have implemented allows exploring a large region of the parameters space, but is also capable of reproducing some aspects of the brain's spontaneous activity (switching between different attractors). The set of attractors we find in some regions of the parameters space share similarities with the different functional modes observed in the resting-state activity [1].

Identification of noisy dynamical systems with parameter estimation based on Hopfield neural networks

An algorithm for estimating time-varying parameters of dynamical systems is proposed, within the large family of prediction error methods. The algorithm is based on the ability of Hopfield neural networks to solve optimisation problems, since its formulation can be summarized as minimisation of the prediction error by means of a continuous Hopfield network. In previous work, it was proved, under mild assumptions, that the estimates converge towards the actual values of parameters and the estimation error remains asymptotically bounded in the presence of measurement noise. The novelty of this work is the advance in the robustness analysis, by considering deterministic disturbances, which do not fulfil the usual statistical hypothesis such as normality and uncorrelatedness. A model of HIV epidemics in Cuba is used as suitable benchmark, which is confirmed by the computation of the sensitivity matrix. The results show a promising performance, in comparison to the conventional Least Squares Estimator. Indeed, the estimation error is almost always lower in the proposed method that in least squares, and it is never significantly higher. Further, from a qualitative point of view, the estimate provided by the Hopfield estimator is smoother, with no overshoot that could eventually destabilize a closed control loop. A significant finding is the fact that the form of the perturbation affects critically the dynamical behaviour and magnitude of the estimation, since the estimation error asymptotically vanishes when the disturbances are additive, but not when they are multiplicative. To summarize, we can conclude that the proposed estimator is an efficient and robust method to estimate time-varying parameters of dynamical systems.

Refined augmented Lagrange Hopfield network-based Lagrange relaxation for hydrothermal scheduling

This paper proposes a refined augmented Lagrange Hopfield network-based Lagrange relaxation (ALHN-LR) for solving short term hydrothermal scheduling (HTS) problem with pumped-storage hydro units. ALHN-LR consists of improved Lagrange relaxation (ILR) and augmented Lagrange Hopfield network (ALHN). For solving the HTS problem, the proposed method applies enhanced ILR for finding thermal unit scheduling, ALHN for solving constrained economic dispatch (CED), and heuristic search-based algorithms in earlier papers for committing hydro and pumped-storage units, repairing ramp rate, emission, and transmission constraint violations, and refining the obtained result. The proposed ALHN which is a continuous Hopfield network with its energy function based on augmented Lagrangian function can properly handle both equality and inequality constraints in CED problem. The proposed ALHN-LR is tested on a hydrothermal system with 17 thermal, two hydro and two pumped-storage units, and the IEEE 24-bus reliability test system over the 168-hour schedule time horizon. Test results indicate that the proposed method can obtain less total costs than those from augmented Hopfield network (AHN), hybrid enhanced Lagrangian relaxation and quadratic programming (hybrid LRQP), and augmented Lagrangian relaxation (ALR).

Real Power Dispatch on Large Scale Power Systems by Augmented Lagrange Hopfield Network

This paper proposes an augmented Lagrange Hopfield network (ALHN) for real power dispatch on large-scale power systems. The proposed ALHN is a continuous Hopfield network with its energy function based on augmented Lagrange function. For this combination, the ALHN method can easily deal with large-scale problems with nonlinear constraints. The proposed ALHN has been tested on systems from 40 units to 240 units, IEEE 118-bus and IEEE 300-bus systems, and the obtained results have been compared to those from other methods. The test results have shown that the ALHN method can obtain better solutions than the compared methods in a very fast manner. Therefore, the proposed ALHN could be favorable for implementation on the real power dispatch problems for large-scale systems.

Augmented Lagrange Hopfield network based Lagrangian relaxation for unit commitment

This paper proposes an augmented Lagrange Hopfield network based Lagrangian relaxation (ALHN-LR) for solving unit commitment (UC) problem with ramp rate constraints. ALHN-LR is a combination of improved Lagrangian relaxation (ILR) and augmented Lagrange Hopfield network (ALHN) enhanced by heuristic search. The proposed ALHN-LR method solves the UC problem in three stages. In the first stage, ILR is used to solve unit scheduling satisfying load demand and spinning reserve constraints neglecting minimum up and down time constraints. In the second stage, heuristic search is applied to refine the obtained unit schedule including primary unit de-commitment, unit substitution, minimum up and down time repairing, and de-commitment of excessive units. In the last stage, ALHN which is a continuous Hopfield network with its energy function based on augmented Lagrangian relaxation is applied to solve constrained economic dispatch (ED) problem and a repairing strategy for ramp rate constraint violations is used if a feasible solution is not found. The proposed ALHN-LR is tested on various systems ranging from 17 to 110 units and obtained results are compared to those from many other methods. Test results indicate that the total production costs obtained by the ALHN-LR method are much less than those from other methods in the literature with a faster manner. Therefore, the proposed ALHN-LR is favorable for large-scale UC implementation.