Neural networks as an approximator for a family of optimization algorithm solutions for online applications

Abstract

In this paper, we propose a sufficient condition at which a neural network can approximate a set of optimization algorithm solutions; we establish under which conditions a neural network can replace an optimization algorithm to solve a problem with the objective of safely deploying that network in a system where online solutions are necessary to simplify the hardware or allowing the processor to solve the optimization problem on time. To that end, first, we define the family of optimization problems to be addressed; then, we construct a vector with the parameters on which the solution depends, in order to propose a function based on the first-order Karush–Kuhn–Tucker conditions to find conditions under which the inverse of the proposed function maps the problem minimizer with respect to the constructed vector, we provide the sufficiency proof of, both, existence and feasibility of approximation by a neural network regarding the inverse function. Two case studies are proposed, one numerical case showing how a neural network can solve an optimization problem faster than popular solvers to illustrate how it can be implemented in applications where the computation time is tight, and the other case is a Model Predictive Control implementation with the optimization problem solver replaced by a neural network which allows a hardware downgrade; both cases are presented with time statistics comparisons.