Evolutionary Design of Long Short Term Memory (LSTM) Ensemble

Abstract

Long Short Term Memory (LSTM) based Recurrent Neural Network is an effective deep learning model with a wide variety of applications. Despite numerous recent success, using a single LSTM network cannot achieve reliable performance on complicated machine learning tasks. An important approach to improve the performance is to learn an ensemble of LSTMs rather than learning a single LSTM. The performance of the ensemble depends on the accuracy and diversity of individual LSTMs. This paper proposes a new evolutionary multi-objective algorithm to automatically design a collection of diverse and accurate LSTMs based on Non-dominated Sort Genetic Algorithm II. We also propose a method to select suitable LSTMs from an evolved non-dominated set of LSTMs to form the ensemble. The obtained results on multiple benchmark classification tasks show that the proposed algorithm can evolve an ensemble of LSTMs, which significantly outperform many state-of-the-art models.