Short-Term Memory Language Model Research Articles

Micro-Blog Sentiment Classification Method Based on the Personality and Bagging Algorithm

Integrated learning can be used to combine weak classifiers in order to improve the effect of emotional classification. Existing methods of emotional classification on micro-blogs seldom consider utilizing integrated learning. Personality can significantly influence user expressions but is seldom accounted for in emotional classification. In this study, a micro-blog emotion classification method is proposed based on a personality and bagging algorithm (PBAL). Introduce text personality analysis and use rule-based personality classification methods to divide five personality types. The micro-blog text is first classified using five personality basic emotion classifiers and a general emotion classifier. A long short-term memory language model is then used to train an emotion classifier for each set, which are then integrated together. Experimental results show that compared with traditional sentiment classifiers, PBAL has higher accuracy and recall. The F value has increased by 9%.

NEWLSTM: An Optimized Long Short-Term Memory Language Model for Sequence Prediction

The long short-term memory (LSTM) model trained on the universal language modeling task overcomes the bottleneck of vanishing gradients in the traditional recurrent neural network (RNN) and shows excellent performance in processing multiple tasks generated by natural language processing. Although LSTM effectively alleviates the vanishing gradient problem in the RNN, the information will be greatly lost in the long distance transmission, and there are still some limitations in its practical use. In this paper, we propose a new model called NEWLSTM, which improves the LSTM model, and alleviates the defects of too many parameters in LSTM and the vanishing gradient. The NEWLSTM model directly correlates the cell state information with current information. The traditional LSTM’s input gate and forget gate are integrated, some components are deleted, the problems of too many LSTM parameters and complicated calculations are solved, and the iteration time is effectively reduced. In this paper, a neural network model is used to identify the relationship between input information sequences to predict the language sequence. The experimental results show that the improved new model is simpler than traditional LSTM models and LSTM variants on multiple test sets. NEWLSTM has better overall stability and can better solve the sparse words problem.

Tailoring an Interpretable Neural Language Model

Neural networks have shown great potential in language modeling. Currently, the dominant approach to language modeling is based on recurrent neural networks (RNNs) and convolutional neural networks (CNNs). Nonetheless, it is not clear why RNNs and CNNs are suitable for the language modeling task since these neural models are lack of interpretability. The goal of this paper is to tailor an interpretable neural model as an alternative to RNNs and CNNs for the language modeling task. This paper proposes a unified framework for language modeling, which can partly interpret the rationales behind existing language models (LMs). Based on the proposed framework, an interpretable neural language model (INLM) is proposed, including a tailored architectural structure and a tailored learning method for the language modeling task. The proposed INLM can be approximated as a parameterized auto-regressive moving average model and provides interpretability in two aspects: component interpretability and prediction interpretability. Experiments demonstrate that the proposed INLM outperforms some typical neural LMs on several language modeling datasets and on the switchboard speech recognition task. Further experiments also show that the proposed INLM is competitive with the state-of-the-art long short-term memory LMs on the Penn Treebank and WikiText-2 datasets.