Abstract
The mathematical foundation of deep learning is the theorem that any continuous function can be approximated within any specified accuracy by using a neural network with certain non-linear activation functions. However, this theorem does not tell us what the network architecture should be and what the values of the weights are. One must train the network to estimate the weights. There is no guarantee that the optimal weights can be reached after training. This paper develops an explicit architecture of a universal deep network by using the Gray code order and develops an explicit formula for the weights of this deep network. This architecture is target function independent. Once the target function is known, the weights are calculated by the proposed formula, and no training is required. There is no concern whether the training may or may not reach the optimal weights. This deep network gives the same result as the shallow piecewise linear interpolation function for an arbitrary target function.
Highlights
As suggested in [1], the idea of using a deep neural network on neuro-dynamics
We develop a formula for the weights of a universal deep network
This network performs piecewiselinear approximation of a one-dimensional (1D) continuous target function f(x) on [a, b]. We extend this deep network to the situations where the target function is d-dimensional
Summary
As suggested in [1], the idea of using a deep neural network on neuro-dynamics. The mainstream research on neural network slowly started after his death in 1971. An early deep learning network algorithm was developed by Ivakhnenko and Lapa in 1967 [2]. They described a deep learning net with 8 layers. Given a training set of input vectors with corresponding target output vectors, layers of additive and multiplicative neuron-like nodes were incrementally grown and trained by regression analysis, pruned with the help of a separate validation set, where regularization was used to weed out superfluous nodes. The numbers of layers and nodes per layer were learned in problem-dependent fashion
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
