Discrete Hopfield Network Research Articles

Binary matrix factorization via collaborative neurodynamic optimization

Binary matrix factorization is an important tool for dimension reduction for high-dimensional datasets with binary attributes and has been successfully applied in numerous areas. This paper presents a collaborative neurodynamic optimization approach to binary matrix factorization based on the original combinatorial optimization problem formulation and quadratic unconstrained binary optimization problem reformulations. The proposed approach employs multiple discrete Hopfield networks operating concurrently in search of local optima. In addition, a particle swarm optimization rule is used to reinitialize neuronal states iteratively to escape from local minima toward better ones. Experimental results on eight benchmark datasets are elaborated to demonstrate the superior performance of the proposed approach against six baseline algorithms in terms of factorization error. Additionally, the viability of the proposed approach is demonstrated for pattern discovery on three datasets.

Balanced clustering based on collaborative neurodynamic optimization

Balanced clustering is a semi-supervised learning approach to data preprocessing. This paper presents a collaborative neurodynamic algorithm for balanced clustering. The balanced clustering problem is formulated as a combinatorial optimization problem and reformulated as an Ising model. A collaborative neurodynamic algorithm is developed to solve the formulated balanced clustering problem based on a population of discrete Hopfield networks or Boltzmann machines reinitialized upon their local convergence by using a particle swarm optimization rule. The algorithm inherits the desirable property of almost-sure convergence of collaborative neurodynamic optimization. Experimental results on six benchmark datasets are elaborated to demonstrate the superior convergence and performance of the proposed algorithm against four existing balanced clustering algorithms in terms of balanced clustering quality.

Multivehicle Task Assignment Based on Collaborative Neurodynamic Optimization With Discrete Hopfield Networks.

This article presents a collaborative neurodynamic optimization (CNO) approach to multivehicle task assignments (TAs). The original combinatorial quadratic optimization problem for TA is reformulated as a quadratic unconstrained binary optimization (QUBO) problem with a quadratic utility function and a penalty function for handling load capacity and cooperation constraints. In the framework of CNO with a population of discrete Hopfield networks (DHNs), a TA algorithm is proposed for solving the formulated QUBO problem. Superior experimental results in four typical multivehicle operation scenarios are reported to substantiate the efficacy of the proposed neurodynamics-based TA approach.

Hash Bit Selection via Collaborative Neurodynamic Optimization With Discrete Hopfield Networks.

Hash bit selection (HBS) aims to find the most discriminative and informative hash bits from a hash pool generated by using different hashing algorithms. It is usually formulated as a binary quadratic programming problem with an information-theoretic objective function and a string-length constraint. In this article, it is equivalently reformulated in the form of a quadratic unconstrained binary optimization problem by augmenting the objective function with a penalty function. The reformulated problem is solved via collaborative neurodynamic optimization (CNO) with a population of classic discrete Hopfield networks. The two most important hyperparameters of the CNO approach are determined based on Monte Carlo test results. Experimental results on three benchmark data sets are elaborated to substantiate the superiority of the collaborative neurodynamic approach to several existing methods for HBS.

Python Application: Visual Approach of Hopfield Discrete Method for Hiragana Images Recognition

Python is a dynamic object-oriented programming language. Python provides strong support for integration with other programming languages and other tools. Python programming is rarely used in the field of artificial intelligence, especially artificial neural networks. This research focuses on running Python programming to recognize hiragana letters. In learning the character of Hiragana, one can experience difficulties because of the many combinations of vowels that form new letters by different means of reading and meaning. Discrete Hopfield network is a fully connected, that every unit is attached to every other unit. This network has asymmetrical weights. At Hopfield Network, each unit has no relationship with itself. Therefore it is expected that a computer system that can help recognize the Hiragana Images. With this pattern recognition Application of Hiragana Images, it is expected the system can be developed further to recognize the Hiragana Images quickly and precisely.

An improved competitive Hopfield network with inhibitive competitive activation mechanism for maximum clique problem

In this paper, we analyze the formula of weights definition in the discrete competitive Hopfield network (DCHOM) and point out its flaw when using it to solve some special instances of maximum clique problem (MCP). Based on the analysis, we propose an improved competitive Hopfield network algorithm (ICHN). In ICHN, we introduce a flexible weight definition method which excites the competitive dynamics, and we also present an initial values setting strategy which efficiently increases the probability of finding optimal solutions. Furthermore, an inhibitive competitive activation mechanism is introduced to form a new input updating rule which reduces significantly the number of neurons with an intermediate level of activations. Our algorithm effectively overcomes the flaw of the DCHOM, and exhibits powerful solving ability for the MCP. Experiments on the benchmark problems and practical applications verify the validity of our algorithm.

Optimisation of underground ventilation systems by means of neural networks

The optimisation of the auxiliary ventilation network of underground mines entails the right choice of fans and duct characteristics. Once the decision-making team has selected the appropriate set of ventilation hardware, the duct line is installed across a main drivage and ends up to the most productive working areas. However, when there are many possible duct line routes from the auxiliary fan to the duct line open end, as is the case in the room and pillar mining method, the choice of a cost effective duct line route, in terms of the total length and bending of the ducts involved, is important for the minimisation of the operating cost of ventilation. In this paper, the problem of finding a cost effective duct line route, among a definite number of possibilities, is tackled with a modified discrete Hopfield network. It is shown that the network provides the optimum result for a trivial case. The implementation of the network on a realistic case is shown to be superior to results from the A* algorithm. However, the feasibility of the proposed solution is highly dependent on mining issues and on the right choice of number and distribution of control points that realise all possible routes from the fan to the headings.

A Hopfield Neural Network for Channel Assignment Problem in Cellular Radio Networks

In wireless mobile communication system, radio spectrum is limited resource. However, efficient use of available channels has been shown to improve the system capacity. The role of a channel assignment scheme is to allocate channels to calls or mobiles in such a way as to minimize call blocking or call dropping probabilities, and also to maximize the quality of service. Channel assignment is known to be an NP-hard optimization problem. In this paper, a new channel-assignment algorithm using a modified Hopfield neural network is proposed. The channel-assignment problem is formulated as an energy-minimization problem that is implemented by a modified discrete Hopfield network. In this algorithm, an energy function is derived, and the appropriate interconnection weights between the neurons are specified. The interconnection weights between the neurons are designed in such a way that each neuron receives inhibitory support if the constraint conditions are violated and receives excitatory support if the constraint conditions are satisfied. The algorithm will be tested by solving seven benchmark problems, where the total number of frequencies varied from 73 to 533. This new algorithm, together with the proposed regular interval initialization and new interconnection weights, has better performance results than the existing algorithms in all of the seven problems.

Blind Detection of 64QAM Signals with a Complex Discrete Hopfield Network

针对复数多电平QAM信号的盲检测问题，该文提出了一个新的复数离散多电平Hopfield神经网络。该网络的实部、虚部各含一个多电平离散激励实函数。该文分析了经典两电平离散Hopfield神经网络能量函数的局限性，构造了一个新的复数多电平神经网的能量函数，并用此能量函数讨论了神经网的稳定性。当该神经网的权矩阵借助接收数据补投影算子构成时，该复数离散多电平Hopfield网络可有效地求解带整数约束的二次规划问题，从而实现QAM信号盲检测。仿真试验表明：该算法所需接收数据较短，就可到达全局真平衡点，计算难度大大降低，具有良好的快速性。

Blind detection algorithm of complex multi-valued discrete Hopfield network

A novel complex multi-valued discrete Hopfield neural network (CMDHNN) is proposed in this paper. A multi-valued discrete activation function and a new energy function for CMDHNN are constructed. The stabilities for multi-valued CMDHNN with asynchronous and synchronous operating modes are also analyzed seperately. The special energy functions own the ability to describe the dynamic characteristics of CMDHNN which the energy functions of existing references cannot explain. Meantime, these energy functions can make the true source signal vector correspond to the minimum point of the energy function of CMDHNN. Furthermore, to verify effectiveness of CMDHNN, the weighted matrix of CMDHNN is constructed by the specific cost function for the blind detection of signals. Simulation results show that the proposed CMDHNN can be used to blindly detect the dense MQAM constellation signals with shorter received signals and the global minimal value of the CMDHNN energy function is verified.

Flexible Two-wheeled Self-balancing Robot Control Based on Hopfield Neural Network

Real control of robot is realized successfully by using spring to imitate the lumbar of human-being and adopt-ing LQR(linear quadratic regulator).For posture control of the flexible two-wheeled self-balancing robot,a kind of discrete Hopfield network way based on associative study is presented,and self-adaptive and self-organizing posture control is accom-plished in a manner of biological control.Firstly,a reasonable energy function is defined for the nonlinear,strong coupling and flexible robot system.Then,a Hopfield neural network controller suitable for this dynamic process is designed based on the dynamics equation of the flexible robot,and its convergence is analyzed.Simulation experiments show the effectiveness of the method and prove the stability of the system.The rationality and availability of the system posture controller design are indicated through analyzing the experiment results in detail.

Discrete Hopfield network combined with estimation of distribution for unconstrained binary quadratic programming problem

Unconstrained binary quadratic programming problem (UBQP) consists in maximizing a quadratic 0-1 function. It is a well known NP-hard problem and is a unified model for a variety of combinatorial optimization problems. This paper presents a discrete Hopfield neural network (DHNN) combined with estimation of distribution algorithm (EDA) for the UBQP. The idea of EDA is combined with the DHNN in order to overcome the local minima problem of the network. Once the network is trapped in local minima, the perturbation based on EDA can generate a new starting point for the DHNN for further search, which is in a promising area characterized by a probability model. Thus, the proposed algorithm, named DHNN–EDA, can escape from local minima and further search better results. The DHNN–EDA is tested on a large number of benchmark problems with size up to 7000 variables. Simulation results on the UBQP show that the DHNN–EDA is better than the other improved DHNN algorithms such as multi-start DHNN and DHNN with random flips, and is better than or competitive with metaheuristic algorithms such as simulated annealing, tabu search, scatter search and memetic algorithm.

On the complexity of enumerating possible dynamics of sparsely connected Boolean network automata with simple update rules

We study how hard is to determine some fundamental properties of dynamics of certain types of network automata. We address the computational complexity of determining how many different possible dynamic evolutions can arise from some structurally very simple, deterministic and sparsely connected network automata. In this as well as our prior, related work, we try to push the limits on the underlying simplicity of two structural aspects of such network automata: (i) the uniform sparseness of their topologies, and (ii) severely restricted local behaviors of the individual agents (that is, the local update rules of the network nodes). In this endeavor, we prove that counting the Fixed Point (FP) configurations and the predecessor and ancestor configurations in two classes of network automata, called Sequential and Synchronous Dynamical Systems (SDSs and SyDSs, respectively), are computationally intractable problems. Moreover, this intractability is shown to hold when each node in such a network is required to update according to (i) a monotone Boolean function, (ii) a symmetric Boolean function, or even (iii) a simple threshold function that is both monotone and symmetric. Furthermore, the hardness of the exact enumeration of FPs and other types of configurations of interest remains to hold even in some severely restricted cases with respect to both the network topology and the diversity (or lack thereof) of individual node's local update rules. Namely, we show that the counting problems of interest remain hard even when the nodes of an SDS or SyDS use at most two different update rules from a given restricted class, and, additionally, when the network topologies are constrained so that each node has only $c = O(1)$ neighbors for small values of constant $c$. Our results also have considerable implications for other discrete dynamical system models studied in applied mathematics, physics, biology and computer science, such as Hopfield networks and spin glasses. In particular, one corollary of our results is that determining the memory capacity of sparse discrete Hopfield networks (viewed as associative memories) remains computationally intractable even when the interconnection and dependence structure among the nodes of a Hopfield network is severely restricted.

Complexity of reachability problems for finite discrete dynamical systems

Sequential Dynamical Systems (SDSs) are a special type of finite discrete dynamical systems that can be used to model simulation systems. We focus on the computational complexity of testing several phase space properties of SDSs. Our main result is a sharp delineation between classes of SDSs whose behavior is easy to predict and those whose behavior is hard to predict. Specifically, we show the following. 1. Several state reachability problems for SDSs are PSPACE-complete, even when restricted to SDSs whose underlying graphs are of bounded bandwidth (and hence of bounded pathwidth and treewidth), and the function associated with each node is symmetric. Moreover, this result holds even when the underlying graph is d-regular for some constant d and all the nodes compute the same symmetric Boolean function. An immediate corollary of this result is a PSPACE-hard lower bound on the complexity of reachability problems for regular generalized 1D-Cellular Automata and undirected systolic networks with Boolean totalistic local transition functions. 2. In contrast, the above reachability problems are solvable in polynomial time for SDSs when the Boolean function associated with each node is symmetric and monotone. The PSPACE-completeness results follow as corollaries of simulation results which show for several classes of SDSs, how one class of SDSs can be efficiently simulated by another (more restricted) class of SDSs. We also prove several structural properties concerning the phase space of an SDS. SDSs are closely related to Cellular Automata (CA), concurrent transition systems, discrete Hopfield networks and systolic networks. This observation in conjunction with our lower bounds for SDSs, yields new PSPACE-hard lower bounds on the complexity of state reachability problems for these models, extending some of the earlier results in [K. Culik II, J. Karhumäki, On totalistic systolic networks, Inform. Process. Lett. 26 (5) (1988) 231–236; P. Floréen, E. Goles, G. Weisbuch, Transient length in sequential iterations of threshold functions, Discrete Appl. Math. 6 (1983) 95–98; P. Floréen, P. Orponen, Complexity issues in discrete Hopfield networks, Research Report No. A-1994-4, Department of Computer Science, University of Helsinki, 1994. Also appears in: I. Parberry (Ed.), Comp. and Learning Complexity of Neural Networks: Advanced Topics, 1999; D. Harel, O. Kupferman, M.Y. Vardi, On the complexity of verifying concurrent transition systems, Inform. and Comput. 173 (2002) 143–161; S.K. Shukla, H.B. Hunt III, D.J. Rosenkrantz, R.E. Stearns, On the complexity of relational problems for finite state processes, in: International Colloquium on Automata Programming and Languages, ICALP, 1996, pp. 466–477; A. Rabinovich, Complexity of equivalence problems for concurrent systems of finite agents, Inform. and Comput. 127 (2) (1997) 164–185].

An Energy Function-Based Design Method for Discrete Hopfield Associative Memory With Attractive Fixed Points

An energy function-based autoassociative memory design method to store a given set of unipolar binary memory vectors as attractive fixed points of an asynchronous discrete Hopfield network (DHN) is presented. The discrete quadratic energy function whose local minima correspond to the attractive fixed points of the network is constructed via solving a system of linear inequalities derived from the strict local minimality conditions. The weights and the thresholds are then calculated using this energy function. If the inequality system is infeasible, we conclude that no such asynchronous DHN exists, and extend the method to design a discrete piecewise quadratic energy function, which can be minimized by a generalized version of the conventional DHN, also proposed herein. In spite of its computational complexity, computer simulations indicate that the original method performs better than the conventional design methods in the sense that the memory can store, and provide the attractiveness for almost all memory sets whose cardinality is less than or equal to the dimension of its elements. The overall method, together with its extension, guarantees the storage of an arbitrary collection of memory vectors, which are mutually at least two Hamming distances away from each other, in the resulting network.

Determination of weights for relaxation recurrent neural networks

A theorem which establishes the solutions of a given optimization problem as stable points in the state space of single-layer relaxation-type recurrent neural networks is proposed. This theorem establishes the necessary conditions for the neural network to converge to a solution by proposing certain values for the constraint weight parameters of the network. Convergence performance of the discrete Hopfield network with the proposed bounds on constraint weight parameters is tested on a set of constraint satisfaction and optimization problems including the traveling salesman problem, the assignment problem, the weighted matching problem, the N-queens problem and the graph path search problem. Simulation and stability analysis results indicate that the set of solutions becomes a subset of the set of stable points in the state space as a result of the suggested bounds. For the cases of the traveling salesman, assignment and weighted matching problems, two sets are equal leading to convergence to a solution after each relaxation. Convergence to a solution after each relaxation is not guaranteed for the N-queens and the graph path search problems since the solution set is a proper subset of the stable point set. Furthermore the simulation results indicate that the discrete Hopfield network converged to mostly average quality solutions as expected from a gradient-descent search algorithm. In conclusion, the suggested bounds on weight parameters guarantee that the discrete Hopfield network will locate a solution after each relaxation for a class of optimization problems of any size, although the solutions will be average quality rather than optimum.

Recurrent Neural Networks for Structural Optimization

This paper presents an improvement for an artificial neural network paradigm that has shown significant potential for successful application to a class of optimization problems in structural engineering. The artificial neural network paradigm includes algorithms that belong to the class of single‐layer, relaxation‐type recurrent neural networks. The suggested improvement enhances the convergence performance and involves a technique that sets the values of weight parameters of the recurrent neural network algorithm. The complete procedure of solving an optimization problem with a single‐layer, relaxation‐type recurrent neural network is introduced. The discrete Hopfield network is employed to solve the weighted matching problem. A set of simulation experiments is performed to analyze the performance of the discrete Hopfield network. Simulation results confirm that the discrete Hopfield network locates a locally optimal solution after each relaxation once the weight parameters are specified as defined in the suggested technique.

Cellular radio channel assignment using a modified Hopfield network

The channel-assignment problem is important in mobile telephone communication. Since the usable range of the frequency spectrum is limited, the optimal channel-assignment problem has become increasingly important. A new channel-assignment algorithm using a modified Hopfield (1985, 1986) neural network is proposed. The channel-assignment problem is formulated as an energy-minimization problem that is implemented by a modified discrete Hopfield network. Also, a new technique to escape the local minima is introduced. In this algorithm, an energy function is derived, and the appropriate interconnection weights between the neurons are specified. The interconnection weights between the neurons are designed in such a way that each neuron receives inhibitory support if the constraint conditions are violated and receives excitatory support if the constraint conditions are satisfied. To escape the local minima, if the number of assigned channels are less than the required channel numbers (RCNs), one or more channels are assigned in addition to already assigned channels such that the total number of assigned channels is the same as the required number of channels in the cell even though the energy is increased. Various initialization techniques, which use the specific characteristics of frequency-assignment problems in cellular radio networks, such as cosite constraint (CSC), adjacent channel constraint (ACC), and cochannel constraint (CCC), and updating methods are investigated. In the previously proposed neural-network approach, some frequencies are fixed to accelerate the convergence time. In our algorithms, no frequency is fixed before the frequency-assignment procedure. This new algorithm, together with the proposed initialization and updating techniques and without fixing frequencies in any cells, has better performance results than the results reported previously utilizing fixed frequencies in certain cells.

A robust boundary-based object recognition in occlusion environment by hybrid Hopfield neural networks

This paper presents a new method of occluded object matching for machine vision applications. The current methods for occluded object matching lack robustness and require high computational effort. In this paper, a new Hybrid Hopfield Neural Network (HHN) algorithm, which combines the advantages of both a Continuous Hopfield Network (CHN) and a Discrete Hopfield Network (DHN), will be described and applied for partially occluded object recognition in a multi-context scenery. The HHN proposed as a new approach provides great fault tolerance and robustness and requires less computation time. Also, advantages of HHN such as reliability and speed will be discussed.

Computational Paradigms in Nanoelectronics: Quantum Coupled Single Electron Logic and Neuromorphic Networks

We describe a new class of nanoelectronic circuits where circuit functions are derived from cooperative, quantum mechanical interactions between single electrons confined in arrays of quantum dots. Two specific architectures are examined: (i) quantum coupled logic in which Boolean logic functions are implemented by quantum mechanical spin-spin interactions between single electrons in arrays of quantum dots, and (ii) quantum neuromorphic networks that exploit the complex spatial and temporal evolution of discrete charge in an ensemble of non-linearly interacting quantum dots to elicit collective computational behavior. The first class of circuits includes combinational and sequential digital systems. Both logically irreversible elements such as half-adders, S-R flip-flops, shift registers, ring counters, etc., and reversible Feynman gates for quantum computation have been designed in this paradigm. These circuits can be endowed with the required “unidirectional” (non-reciprocal) character that previous (flawed) designs of similar circuits lacked. The second class of circuits comprises discrete Hopfield networks which utilize single electron tunneling events in arrays of metallic islands to perform neuromorphic computation. They can solve NP-complete optimization problems (such as the traveling salesman problem), produce associative memory effects and also exhibit rudimentary image-processing capability.