Solution Existence Theory for Artificial Neural Networks

Abstract

Artificial neural networks (NNs) have been widely used in creation of various machine learning (ML) models. Training an NN for a given dataset is essentially finding a set of weights and biases, known as learning (or training) parameters. An NN uses the affine transformations to encode the features in the dataset in the form of learning parameters. One may naturally ask fundamental questions: (1) can the NN be trained, meaning that can a solution for a set of learning parameters can be found? and (2) is the solution unique? This paper addresses these questions. First, detailed formulation is presented for a simple NN with just one basic affine transformation unit (ATU) for linear regression problems. Conditions for solution existence and uniqueness will be examined, leading to the Solution Existence Theory. A discussion will be presented when this Solution Existence Theory is extended to general NNs for complex problems, in addition to a discussion on predictability of NNs.