Shallow Network Training With Dynamic Sample Weights Decay - a Potential Function Approximator for Reinforcement Learning

Abstract

Neural Networks are commonly used as function approximators in Reinforcement Learning, and the Extreme Learning Machine is one of the best algorithms to quickly train a shallow network. The online and sequential version OS-ELM could be a great candidate to quickly train a network to be a function approximator for Reinforcement Learning, but due to its non-forgetting properties it is actually not suitable for direct use with value estimations that improve in accuracy over time. This paper presents an alternative Neural Network training algorithm based on OS-ELM, which is able to perform learning online while dynamically modifying the weights of previously learned samples in order to decrease the importance of old samples learned over time. A mathematical derivation of the formulas used is presented, along with results of experiments showing equivalence of our algorithm to ELM when learning classic datasets and the advantage provided when dealing with Reinforcement Learning data.