Immunocomputing-Based Approach for Optimizing the Topologies of LSTM Networks

Abstract

This paper aims to automatically design optimal LSTM topologies using the clonal selection algorithm (CSA) to solve text classification tasks such as sentiment analysis and SMS spam classification. Designing optimal topologies involves determining the best configuration of hyperparameters that will give the best performance. The current state-of-the-art LSTM topologies are often designed using trial and error approaches which are incredibly time-consuming and require domain experts. Our proposed method, referred to as CSA-LSTM, is evaluated using the Large Movie Review Dataset (IMDB). Furthermore, to verify the robustness of the hyperparameters discovered by CSA for the IMDB dataset, we have used them for the other datasets, viz. the Twitter US Airline Sentiment and the SMS Spam Collection. Additionally, the discovered hyperparameters for the LSTM are combined with pre-determined convolutional neural network (CNN) layers to achieve the same or better results to fast the training time and fewer trainable parameters. For further verification and evaluation of the generalization ability and effectiveness of the proposed approach, it is compared with four machine learning algorithms widely used for text classification tasks: (1) random forest (RF), (2) logistic regression (LR), (3) support vector machine (SVM), and (4) multinomial naive Bayes (NB). The results of our experiments show that the LSTM topologies automatically designed by our CSA method are less expensive, reusable and outperform the machine learning algorithms and other models in the literature evaluated on the same three datasets. Through our proposed method, LSTM's best topology can be self-determined without any human intervention, making CSA-based algorithms a promising approach to automatically design optimal LSTM topologies that provide the best performance for a given task.