Multilayer perceptron for nonlinear programming

Abstract

A new method for solving nonlinear programming problems within the framework of a multilayer neural network perceptron is proposed. The method employs the Penalty Function method to transform a constrained optimization problem into a sequence of unconstrained optimization problems and then solves the sequence of unconstrained optimizations of the transformed problem by training a series of multilayer perceptrons. The neural network formulation is represented in such a way that the multilayer perceptron prediction error to be minimized mimics the objective function of the unconstrained problem, and therefore, the minimization of the objective function for each unconstrained optimization is attained by training a single perceptron. The multilayer perceptron allows for the transformation of problems with two-sided bounding constraints on the decision variables x , e.g., a⩽ x n ⩽ b, into equivalent optimization problems in which these constraints do not explicitly appear. Hence, when these are the only constraints in the problem, the transformed problem is constraint free (i.e., the transformed objective function contains no penalty terms) and is solved by training a multilayer perceptron only once. In addition, we present a new Penalty Function method for solving nonlinear programming problems that is parameter free and guarantees that feasible solutions are obtained when the optimal solution is on the boundary of the feasible region. Simulation results, including an example from operations research, illustrate the proposed methods. Scope and purpose In this article, the multilayer perceptron, the most widely used type of artificial neural network and also known as the multilayer feedforward neural network, is applied in the development of a new method for solving nonlinear constrained optimization problems. In addition to providing a new application for the multilayer perceptron, this paper also shows that for a certain class of constrained optimization problems the proposed neural network methodology is more efficient than conventional optimization methods. Another contribution of this article is the description of a new variation of the Penalty Function method for solving nonlinear programming problems that combines the advantages of the Exterior Penalty Function and the Interior (Barrier) Penalty Function. Examples using an analytical function and real world data are used to illustrate the advantages of the proposed methods. The paper by Humelhart et al. [5] provides a mathematical description of the multilayer perceptron and a method for training it, and the book by Avriel [6] contains a chapter on Penalty Function methods for solving nonlinear constrained optimization problems.