A neural network theory for constrained optimization

Abstract

A variety of real-world problems can be formulated into continuous optimization problems with constraint equalities. The real-world problem here can include, for example, the traveling salesman problem, the Dido’s isoperimetric problem, the Hitchcock’s transportation problem, the network flow problem and the associative memory problem. In spite of the significance, there has not yet been developed any robust solving method that works efficiently across a broad spectrum of optimization problems. The recent Hopfield’s neural network method to solve the traveling salesman problem is a potentially promising candidate because of the efficiency due to its parallel processing. His method, however, has certain drawbacks that must be removed away before it can be qualified for an efficient, robust solving method. That is: (a) locally minimum solutions instead of globally minimum; (b) possible infeasible solutions; (c) heuristic choice of network parameters and an initial state; (d) quadratic objective functions instead of arbitrary nonlinear objective functions with arbitrary nonlinear equality constraints; and (e) unorganized mathematical formulation of the network for extension. This paper develops from the Hopfield method an efficient, robust network solving method of the continuous optimization problem with constraint equalities that resolves all the drawbacks except for (a) that has already been resolved by others. The development is mathematically rigorous and thus constitutes a solid foundation of a neural network theory for constrained optimization.