Baseline Long Short-term Memory Research Articles

Intelligent Abstractive Summarization of Scholarly Publications with Transfer Learning

Intelligent abstractive text summarization of scholarly publications refers to machine-generated summaries that capture the essential ideas of an article while maintaining semantic coherence and grammatical accuracy. As information continues to grow at an overwhelming rate, text summarization has emerged as a critical area of research. In the past, summarization of scientific publications predominantly relied on extractive methods. These approaches involve selecting key sentences or phrases directly from the original document to create a summary or generate a suitable title. Although extractive methods preserve the original wording, they often lack the ability to produce a coherent, concise, and fluent summary, especially when dealing with complex or lengthy texts. In contrast, abstractive summarization represents a more sophisticated approach. Rather than extracting content from the source, abstractive models generate summaries using new language, often incorporating words and phrases not found in the original text. This allows for more natural, human-like summaries that better capture the key ideas in a fluid and cohesive manner. This study introduces two advanced models for generating titles from the abstracts of scientific articles. The first model employs a Gated Recurrent Unit (GRU) encoder coupled with a greedy-search decoder, while the second utilizes a Transformer model, known for its capacity to handle long-range dependencies in text. The findings demonstrate that both models outperform the baseline Long Short-Term Memory (LSTM) model in terms of efficiency and fluency. Specifically, the GRU model achieved a ROUGE-1 score of 0.2336, and the Transformer model scored 0.2881, significantly higher than the baseline LSTM model, which reported a ROUGE-1 score of 0.1033. These results underscore the potential of abstractive models to enhance the quality and accuracy of summarization in academic and scholarly contexts, offering more intuitive and meaningful summaries.

Semi-supervised prediction method for time series based on Monte Carlo and time fusion feature attention

Accurate and reliable forecasting of time sequences is a challenging task in cyber–physical systems (CPS). Traditional time series prediction models struggle to provide accurate predictions for diverse types of time series data, especially with missing data due to variations in scale, types, and the complexity of real-world production environments. In this paper, we introduce a hybrid model named Temporal Feature Fusion Attention-based Monte Carlo Semi-supervised Long Short Term Memory (LSTM) network to address this issue. The model encodes the current state and historical information using a current time feature state vector. It then calculates the hidden feature vectors for the time series at different time points (past, present, and future), as well as for the current Monte Carlo filtering sequences. This approach leverages the correlation of time series features, transfers and fuses crucial historical features within the sequence with the optimized sequence feature information from the Monte Carlo algorithm. Our experiments confirm that with 10% of labeled data missing, our proposed method significantly improves the evaluation metric Mean Absolute Percentage Error (MAPE) by 17.827% compared to the baseline LSTM model. Moreover, our method surpasses other state-of-the-art methods across four distinct time series datasets, achieving the best prediction results.

Optimizing energy consumption in smart homes using GA-LSTM

The need to optimize energy consumption arises from the inadequate energy supply many homes face. However, to optimize energy consumption in a home, one must be equipped with the knowledge of the energy consumption rate and energy supply rate in the home. This paper proposed the use of a Long Short-Term Memory (LSTM) model optimized by Genetic Algorithm (GA) to optimize the energy consumption in a smart home. The model was designed using 8 input variables, which were observed weather information of a given region over a span of 350 days. The data set was split into a training data set and a test data set in the ratio 4:1. Both the baseline LSTM model and our proposed GA-LSTM model was used to train and test the data. The results of the baseline LSTM model showed a Mean Average Percentage Error (MAPE) of 0.15 and a Root Mean Square Error (RMSE) of 0.0007, while our proposed GA-LSTM recorded an MAPE of 0.07 and RMSE of 0.0004. The implication of these results is a better performance of our proposed GA-LSTM model over the baseline LSTM model. This paper demonstrated how the Genetic Algorithm was able to optimize the baseline LSTM model by selecting the best weights during the LSTM training process. The findings of this paper are recommended for smart homes having the capability to monitor energy usage of installed electrical appliances and the ability to predict to weather condition of the region the home is sited.

Fault detection in automated production systems based on a long short-term memory autoencoder

Abstract In this paper, a hybrid model of regularized Long Short-Term Memory (LSTM) and autoencoder for fault detection in automated production systems is proposed. The presented LSTM autoencoder is used as a stochastic process model, which captures the normal behavior of a production system and allows to predict the probability distribution of sensor data. Discrepancies between the observed sensor data and the predicted probability density distribution are detected as potential faults. The approach combines the advantages of LSTMs and autoencoders: The correlations between individual sensor signals are exploited by an autoencoder, while the temporal dependencies are captured by LSTM neurons. A key challenge in training such a process model from historical data is to control the information passed through the latent space of the autoencoder. Different regularization methods are investigated for this purpose. Fault detection with the proposed LSTM autoencoder has been evaluated on the use case of an industrial penicillin production, achieving significantly improved results in comparison to the baseline LSTM.

Deep causal speech enhancement and recognition using efficient long-short term memory Recurrent Neural Network.

Long short-term memory (LSTM) has been effectively used to represent sequential data in recent years. However, LSTM still struggles with capturing the long-term temporal dependencies. In this paper, we propose an hourglass-shaped LSTM that is able to capture long-term temporal correlations by reducing the feature resolutions without data loss. We have used skip connections in non-adjacent layers to avoid gradient decay. In addition, an attention process is incorporated into skip connections to emphasize the essential spectral features and spectral regions. The proposed LSTM model is applied to speech enhancement and recognition applications. The proposed LSTM model uses no future information, resulting in a causal system suitable for real-time processing. The combined spectral feature sets are used to train the LSTM model for improved performance. Using the proposed model, the ideal ratio mask (IRM) is estimated as a training objective. The experimental evaluations using short-time objective intelligibility (STOI) and perceptual evaluation of speech quality (PESQ) have demonstrated that the proposed model with robust feature representation obtained higher speech intelligibility and perceptual quality. With the TIMIT, LibriSpeech, and VoiceBank datasets, the proposed model improved STOI by 16.21%, 16.41%, and 18.33% over noisy speech, whereas PESQ is improved by 31.1%, 32.9%, and 32%. In seen and unseen noisy situations, the proposed model outperformed existing deep neural networks (DNNs), including baseline LSTM, feedforward neural network (FDNN), convolutional neural network (CNN), and generative adversarial network (GAN). With the Kaldi toolkit for automated speech recognition (ASR), the proposed model significantly reduced the word error rates (WERs) and reached an average WER of 15.13% in noisy backgrounds.

Abstract 15632: Warfarin Dose Management Using Offline Deep Reinforcement Learning

Introduction: Warfarin is a commonly prescribed anticoagulant for treating atrial fibrillation, mechanical valves, and venous thromboembolism. Warfarin dose management remains challenging due to dosing variability between patients and warfarin’s narrow therapeutic window. Time in therapeutic range (TTR) is critical to warfarin’s safety and efficacy, but TTR typically remains low (40-50%) in community practices. Specialized anticoagulation clinics and protocolized approaches can increase TTR but have great administrative burdens and health care costs. Aim: To develop standardized optimal warfarin dose decision support using a machine learning model based on time series anticoagulation data and patient demographic characteristics Methods: The dataset included 12,497 warfarin patients monitored in the anticoagulation tracker of electronic medical records across the Mayo Clinic Enterprise. The dataset contained time series anticoagulation data (warfarin dose, INR, INR target range) and patient demographic characteristics. We implemented an offline deep reinforcement learning model (DRL) to predict the cumulative warfarin dose for the days until the next INR test based on a patient’s historical anticoagulation data. Prior approaches utilized traditional supervised learning methods such as regression or Long Short Term Memory (LSTM) neural networks to approximate a function mimicking the behavior of the training data. On the other hand, DRL learned an optimal dosing policy through continuous interaction and feedback from the training data. The key advantage of DRL is the model can learn to behave differently (and potentially better) for suboptimal clinical states in the data such as overdosing or underdosing. To evaluate the DRL model we compared the predicted warfarin doses with the physician-prescribed doses Results: DRL model’s prediction accuracy was 96.96%, outperforming our implementation of a baseline LSTM model with a prediction accuracy of 70.58%. Further evaluation of the DRL model indicated that the model correctly adjusted the warfarin dose at time steps when patients had out-of-range INRs. Conclusions: Offline deep reinforcement learning demonstrates potential in supporting warfarin dose management to maximize TTR.

Improving Indoor Occupancy Detection Accuracy of the SLEEPIR Sensor Using LSTM Models

We recently developed a synchronized low energy electronically chopped passive infrared (SLEEPIR) sensor node to detect stationary and moving occupants. It uses a liquid crystal shutter to modulate the infrared signal received by a traditional passive infrared (PIR) sensor and thus enables its capability to detect stationary occupants. However, the detection accuracy of the SLEEPIR sensor can be easily influenced by infrared environmental disturbances. To address this problem, in this paper, we propose two Long Short-Term Memory (LSTM) models to filter infrared environmental disturbance, named baseline LSTM (Base.LSTM) and statistical LSTM (Stat.LSTM). They use the sensor node raw output and statistical features as their respective input. For comparison, we propose two other models: the occupancy state switch detection (SSD) algorithm that directly uses a predetermined threshold voltage value to classify the occupancy state and its status change; and the multi-layer perception (MLP) classifier with statistical feature inputs (Stat.ML). To validate their detection performance, we designed two testing scenarios in different environment settings: (i) Daily occupancy tests and (ii) EDGE case tests. The first scenario intends to restore complex real-life environmental situations as much as possible in the lab and apartment rooms. The second scenario aims to verify their detection accuracy under different environmental temperatures. This scenario also considers different occupancy postures, such as lying down. Experimental results show that the detection accuracy of both LSTM models (> 95%) in both testing scenarios outperforms that of the SSD (around 82% to 94%) and the Stat.ML(around 80% to 90%).

A Cryptocurrency Price Prediction Model using Deep Learning

Cryptocurrencies have gained immense popularity in recent years as an emerging asset class, and their prices are known to be highly volatile. Predicting cryptocurrency prices is a difficult task due to their complex nature and the absence of a central authority. In this paper, our proposal is to employ Long Short-Term Memory (LSTM) networks, a type of deep learning technique to forecast the prices of cryptocurrencies. We use historical price data and technical indicators as inputs to the LSTM model, which learns the underlying patterns and trends in the data. To improve the accuracy of the predictions, we also incorporate a Change Point Detection (CPD) technique using the Pruned Exact Linear Time (PELT) algorithm. This method allows us to detect significant changes in cryptocurrency prices and adjust the LSTM model accordingly, leading to better predictions. We evaluate our approach predominantly on Bitcoin cryptocurrency, but the model can be implemented on other cryptocurrencies provided there are valid historical price data. Our experimental results show that our proposed model outperforms the baseline LSTM algorithm, achieving higher accuracy and better performance in terms of Mean Absolute Error (MAE), Mean Square Error (MSE), and Root Mean Square Error (RMSE). Our research findings suggest that combining deep learning techniques such as LSTM with change point detection techniques such as PELT can improve cryptocurrency price prediction accuracy and have practical implications for investors, traders, and financial analysts.

Recognising Cattle Behaviour with Deep Residual Bidirectional LSTM Model Using a Wearable Movement Monitoring Collar

Cattle behaviour is a significant indicator of cattle welfare. With the advancements in electronic equipment, monitoring and classifying multiple cattle behaviour patterns is becoming increasingly important in precision livestock management. The aim of this study was to detect important cattle physiological states using a neural network model and wearable electronic sensors. A novel long short-term memory (LSTM) recurrent neural network model that uses two-way information was developed to accurately classify cattle behaviour and compared with baseline LSTM. Deep residual bidirectional LSTM and baseline LSTM were used to classify six behavioural patterns of cows with window sizes of 64, 128 and 256 (6.4 s, 12.8 s and 25.6 s, respectively). The results showed that when using deep residual bidirectional LSTM with window size 128, four classification performance indicators, namely, accuracy, precision, recall, and F1-score, achieved the best results of 94.9%, 95.1%, 94.9%, and 94.9%, respectively. The results showed that the deep residual bidirectional LSTM model can be used to classify time-series data collected from twelve cows using inertial measurement unit collars. Six aim cattle behaviour patterns can be classified with high accuracy. This method can be used to quickly detect whether a cow is suffering from bovine dermatomycosis. Furthermore, this method can be used to implement automated and precise cattle behaviour classification techniques for precision livestock farming.

An Acoustic Feature-Based Deep Learning Model for Automatic Thai Vowel Pronunciation Recognition

For Thai vowel pronunciation, it is very important to know that when mispronunciation occurs, the meanings of words change completely. Thus, effective and standardized practice is essential to pronouncing words correctly as a native speaker. Since the COVID-19 pandemic, online learning has become increasingly popular. For example, an online pronunciation application system was introduced that has virtual teachers and an intelligent process of evaluating students that is similar to standardized training by a teacher in a real classroom. This research presents an online automatic computer-assisted pronunciation training (CAPT) using deep learning to recognize Thai vowels in speech. The automatic CAPT is developed to solve the inadequacy of instruction specialists and the complex vowel teaching process. It is a unique system that develops computer techniques integrated with linguistic theory. The deep learning model is the most significant part of recognizing vowels pronounced for the automatic CAPT. The major challenge in Thai vowel recognition is the correct identification of Thai vowels when spoken in real-world situations. A convolutional neural network (CNN), a deep learning model, is applied and developed in the classification of pronounced Thai vowels. A new dataset for Thai vowels was designed, collected, and examined by linguists. The result of an optimal CNN model with Mel spectrogram (MS) achieves the highest accuracy of 98.61%, compared with Mel frequency cepstral coefficients (MFCC) with the baseline long short-term memory (LSTM) model and MS with the baseline LSTM model have an accuracy of 94.44% and 90.00% respectively.

Prediction of Glacially Derived Runoff in the Muzati River Watershed Based on the PSO-LSTM Model

The simulation and prediction of glacially derived runoff are significant for water resource management and sustainable development in water-stressed arid regions. However, the application of a hydrological model in such regions is typically limited by the intricate runoff production mechanism, which is associated with snow and ice melting, and sparse monitoring data over glacierized headwaters. To address these limitations, this study develops a set of mathematical models with a certain physical significance and an efficient particle swarm optimization algorithm by applying long- and short-term memory networks on the glacierized Muzati River basin. First, the trends in the runoff, precipitation, and air temperature are analyzed from 1990 to 2015, and differences in their correlations in this period are exposed. Then, Particle Swarm Optimization–Long Short-Term Memory (PSO-LSTM) and Bi-directional Long Short-Term Memory (BiLSTM) models are combined and applied to the precipitation and air temperature data to predict the glacially derived runoff. The prediction accuracy is validated by the observed runoff at the river outlet at the Pochengzi hydrological station. Finally, two other types of models, the RF (Random Forest) and LSTM (Long Short-Term Memory) models, are constructed to verify the prediction results. The results indicate that the glacially derived runoff is strongly correlated with air temperature and precipitation. However, in the study region over the past 26 years, the air temperature was not obviously increasing, and the precipitation and glacially derived runoff were significantly decreasing. The test results show that the PSO-LSTM and BiLSTM runoff prediction models perform better than the RF and LSTM models in the glacierized Muzati River basin. In the validation period, among all models, the PSO-LSTM model has the smallest mean absolute error and root-mean-square error and the largest coefficient of determination of 6.082, 8.034, and 0.973, respectively. It is followed by the BiLSTM model having a mean absolute error, root-mean-square error, and coefficient of determination of 6.751, 9.083, and 0.972, respectively. These results imply that both the particle swarm optimization algorithm and the bi-directional structure can effectively enhance the prediction accuracy of the baseline LSTM model. The results presented in this study can provide a deeper understanding and a more appropriate method of predicting the glacially derived runoff in glacier-fed river basins.

Water Quality Prediction for Smart Aquaculture Using Hybrid Deep Learning Models

Water quality prediction (WQP) plays an essential role in water quality management for aquaculture to make aquaculture production profitable and sustainable. In this work, we propose hybrid deep learning (DL) models, convolutional neural network (CNN) with the long short-term memory (LSTM) and gated recurrent unit (GRU) for aquaculture WQP. CNN can effectively fetch the aquaculture water quality characteristics, whereas GRU and LSTM can learn long-term dependencies in the time series data. We conduct experiments using the two different water quality datasets and present an extensive study on the impact of hyperparameters on the performance of the proposed hybrid DL models. Furthermore, the performance of hybrid CNN-LSTM and CNN-GRU models are compared with different baseline LSTM, GRU and CNN DL models and also with attention-based LSTM and attention-based GRU DL models. The results show that the hybrid CNN-LSTM outperformed all other models in terms of prediction accuracy and computation time.

LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes.

Human Activity Recognition (HAR) employing inertial motion data has gained considerable momentum in recent years, both in research and industrial applications. From the abstract perspective, this has been driven by an acceleration in the building of intelligent and smart environments and systems that cover all aspects of human life including healthcare, sports, manufacturing, commerce, etc. Such environments and systems necessitate and subsume activity recognition, aimed at recognizing the actions, characteristics, and goals of one or more individuals from a temporal series of observations streamed from one or more sensors. Due to the reliance of conventional Machine Learning (ML) techniques on handcrafted features in the extraction process, current research suggests that deep-learning approaches are more applicable to automated feature extraction from raw sensor data. In this work, the generic HAR framework for smartphone sensor data is proposed, based on Long Short-Term Memory (LSTM) networks for time-series domains. Four baseline LSTM networks are comparatively studied to analyze the impact of using different kinds of smartphone sensor data. In addition, a hybrid LSTM network called 4-layer CNN-LSTM is proposed to improve recognition performance. The HAR method is evaluated on a public smartphone-based dataset of UCI-HAR through various combinations of sample generation processes (OW and NOW) and validation protocols (10-fold and LOSO cross validation). Moreover, Bayesian optimization techniques are used in this study since they are advantageous for tuning the hyperparameters of each LSTM network. The experimental results indicate that the proposed 4-layer CNN-LSTM network performs well in activity recognition, enhancing the average accuracy by up to 2.24% compared to prior state-of-the-art approaches.

Improving Recurrent Neural Network Responsiveness to Acute Clinical Events

Predictive models in acute care settings must immediately recognize precipitous changes in a patient's status when presented with data reflecting such changes. Recurrent neural networks (RNN) have become popular for clinical decision support models but exhibit a delayed response to acute events. New information must propagate through the RNN's cell state before the total impact is reflected in the model's predictions. Input data perseveration is a method to train more responsive RNN-based models. Input data is replicated k times during training and deployment. Each replication propagates through the cell state and output of the RNN, but only the output at the final replication is maintained and broadcast as the prediction for evaluation. De-identified Electronic Medical Records (EMR) of 12, 826 patients admitted to a tertiary care pediatric academic center between 01/2009-02/2019 were analyzed. A baseline Long Short-Term Memory (LSTM) model ( k=1), four LSTMs with increasing amounts of input data perseveration ( k=2 to k=5), and an LSTM with an attention mechanism were trained to predict ICU-mortality. Performance of models was compared using Area Under the Receiver Operating Characteristic Curve (AUROC) after increasing periods of observation from one to 12 hours. The average variation of the change in predicted mortality immediately following defined acute events measured responsiveness. The AUROC gains due to input perseveration were larger at the earlier times of prediction ( ≤ 6 hours), increasing at the first hour from 0.77 with no input data perseveration to 0.83 when k=5. An LSTM with k=5 was $2-3$ times more responsive to acute events than a baseline LSTM.

Convolutional long short-term memory model for recognizing construction workers’ postures from wearable inertial measurement units

This paper proposes using Deep Neural Networks (DNN) models for recognizing construction workers’ postures from motion data captured by wearable Inertial Measurement Units (IMUs) sensors. The recognized awkward postures can be linked to known risks of Musculoskeletal Disorders among workers. Applying conventional Machine Learning (ML)-based models has shown promising results in recognizing workers’ postures. ML models are limited – they reply on heuristic feature engineering when constructing discriminative features for characterizing postures. This makes further improving the model performance regarding recognition accuracy challenging. In this paper, the authors investigate the feasibility of addressing this problem using a DNN model that, through integrating Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM) layers, automates feature engineering and sequential pattern detection. The model’s recognition performance was evaluated using datasets collected from four workers on construction sites. The DNN model integrating one convolutional and two LSTM layers resulted in the best performance (measured by F1 Score). The proposed model outperformed baseline CNN and LSTM models suggesting that it leveraged the advantages of the two baseline models for effective feature learning. It improved benchmark ML models’ recognition performance by an average of 11% under personalized modelling. The recognition performance was also improved by 3% when the proposed model was applied to 8 types of postures across three subjects. These results support that the proposed DNN model has a high potential in addressing challenges for improving the recognition performance that was observed when using ML models.

LSTM-Based Battery Remaining Useful Life Prediction With Multi-Channel Charging Profiles

Remaining useful life (RUL) prediction of lithium-ion batteries can reduce the risk of battery failure by predicting the end of life. In this paper, we propose novel RUL prediction techniques based on long short-term memory (LSTM). To estimate RUL even in the presence of capacity regeneration phenomenon, we consider multiple measurable data from battery management system such as voltage, current and temperature charging profiles whose patterns vary as aging. Unlike the traditional LSTM prediction that matches input layer with output layer as one-to-one structure, we leverage many-to-one structure to be flexible for various input types and to substantially reduce the number of parameters for better generalization. Using the NASA lithium-ion battery datasets, we verify the accuracy of the proposed LSTM-based RUL prediction. The experimental results show that the proposed single-channel LSTM model improves the mean absolute percentage error (MAPE) by 39.2% compared to the baseline LSTM model. Furthermore, the proposed multi-channel LSTM model significantly improves the MAPE, e.g., by 63.7% compared to the baseline; the proposed model achieves 0.47–1.88% of MAPE while the state-of-the-art baseline LSTM shows 0.6–6.45% of MAPE.

Harmonic Loss Function for Sensor-Based Human Activity Recognition Based on LSTM Recurrent Neural Networks

Human activity recognition (HAR) has been a very popular field in both real practice and theoretical research. Over the years, a number of many-vs-one Long Short-Term Memory (LSTM) models have been proposed for the sensor-based HAR problem. However, how to utilize sequence outputs of them to improve the HAR performance has not been studied seriously. To solve this problem, we present a novel loss function named harmonic loss, which is utilized to improve the overall classification performance of HAR based on baseline LSTM networks. First, label replication method is presented to duplicate true labels at each sequence step in many-vs-one LSTM networks, thus each sequence step can generate a local error and a local output. Then, considering the importance of different local errors and inspired by the Ebbinghaus memory curve, the harmonic loss is proposed to give unequal weights to different local errors based on harmonic series equation. Additionally, to improve the overall classification performance of HAR, integrated methods are utilized to exploit the sequence outputs of LSTM models based on harmonic loss and ensemble learning strategy. Finally, based on the LSTM model construction and hyper-parameter setting, extensive experiments are conducted. A series of experimental results demonstrate that our harmonic loss significantly achieves higher macro-F1 and accuracy than strong baselines on two public HAR benchmarks. Compared with previous state-of-art methods, our proposed methods can achieve competitive classification performance.

Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach

With the advent of Big Data, nowadays in many applications databases containing large quantities of similar time series are available. Forecasting time series in these domains with traditional univariate forecasting procedures leaves great potentials for producing accurate forecasts untapped. Recurrent neural networks (RNNs), and in particular Long Short Term Memory (LSTM) networks, have proven recently that they are able to outperform state-of-the-art univariate time series forecasting methods in this context, when trained across all available time series. However, if the time series database is heterogeneous, accuracy may degenerate, so that on the way towards fully automatic forecasting methods in this space, a notion of similarity between the time series needs to be built into the methods. To this end, we present a prediction model that can be used with different types of RNN models on subgroups of similar time series, which are identified by time series clustering techniques. We assess our proposed methodology using LSTM networks, a widely popular RNN variant, together with various clustering algorithms, such as kMeans, DBScan, Partition Around Medoids (PAM), and Snob. Our method achieves competitive results on benchmarking datasets under competition evaluation procedures. In particular, in terms of mean sMAPE accuracy it consistently outperforms the baseline LSTM model, and outperforms all other methods on the CIF2016 forecasting competition dataset.

Emphasizing Essential Words for Sentiment Classification Based on Recurrent Neural Networks

With the explosion of online communication and publication, texts become obtainable via forums, chat messages, blogs, book reviews and movie reviews. Usually, these texts are much short and noisy without sufficient statistical signals and enough information for a good semantic analysis. Traditional natural language processing methods such as Bow-of-Word (BOW) based probabilistic latent semantic models fail to achieve high performance due to the short text environment. Recent researches have focused on the correlations between words, i.e., term dependencies, which could be helpful for mining latent semantics hidden in short texts and help people to understand them. Long short-term memory (LSTM) network can capture term dependencies and is able to remember the information for long periods of time. LSTM has been widely used and has obtained promising results in variants of problems of understanding latent semantics of texts. At the same time, by analyzing the texts, we find that a number of keywords contribute greatly to the semantics of the texts. In this paper, we establish a keyword vocabulary and propose an LSTM-based model that is sensitive to the words in the vocabulary; hence, the keywords leverage the semantics of the full document. The proposed model is evaluated in a short-text sentiment analysis task on two datasets: IMDB and SemEval-2016, respectively. Experimental results demonstrate that our model outperforms the baseline LSTM by 1%~2% in terms of accuracy and is effective with significant performance enhancement over several non-recurrent neural network latent semantic models (especially in dealing with short texts). We also incorporate the idea into a variant of LSTM named the gated recurrent unit (GRU) model and achieve good performance, which proves that our method is general enough to improve different deep learning models.