Proposed Long Short-term Memory Model Research Articles

Efficient Long Short-Term Memory-Based Sentiment Analysis of E-Commerce Reviews.

In today's modern era, e-commerce is making headway through the process of bringing goods within everyone's grasp. Consumers are not even required to step out of the comfort of their homes for buying things, which makes it very convenient for them. Moreover, there is a wide variety of brands to choose from. Since more customers depend on online shopping platforms these days, the value of ratings is also growing. To buy these products, people rely solely on the reviews that are being provided about the products. To analyze these reviews, sentiment analysis needs to be performed, which can prove useful for both the buyers and the manufacturer. This paper describes the process of sentiment analysis and its requirements. In this paper, Amazon Review dataset 2018 has been used for carrying out our research and Long Short-Term Memory (LSTM) has been combined with word2vec representation, resulting in improving the overall performance. A gating mechanism was used by LSTM during the training process. The proposed LSTM model was evaluated on four performance measures: accuracy, precision, recall, and F1 score, and achieved overall higher results when compared with other baseline models.

Machine Learning-Based Cardiac Arrest Prediction for Early Warning System

The early warning system detects early and responds quickly to emergencies in high-risk patients, such as cardiac arrest in hospitalized patients. However, traditional early warning systems have the problem of frequent false alarms due to low positive predictive value and sensitivity. We conducted early prediction research on cardiac arrest using time-series data such as biosignal and laboratory data. To derive the data attributes that affect the occurrence of cardiac arrest, we performed a correlation analysis between the occurrence of cardiac arrest and the biosignal data and laboratory data. To improve the positive predictive value and sensitivity of early cardiac arrest prediction, we evaluated the performance according to the length of the time series of measured biosignal data, laboratory data, and patient data range. We propose a machine learning and deep learning algorithm: the decision tree, random forest, logistic regression, long short-term memory (LSTM), gated recurrent unit (GRU) model, and the LSTM–GRU hybrid model. We evaluated cardiac arrest prediction models. In the case of our proposed LSTM model, the positive predictive value was 85.92% and the sensitivity was 89.70%.

InSAR Data Analysis using Deep Neural Networks

Among all natural disasters, earthquakes cause the most severe devastation to both infrastructure as well as human lives. The pattern of surface deformation from satellite-based measures, such as interferometric synthetic aperture radar (InSAR), helps understand crustal dynamics and associated seismic hazards in tectonically active areas. In the last few years, the exponentially rising volume of InSAR data has led to the formulation of automatic crustal deformation algorithms through data-intensive machine learning models. In this study, we propose the long-short-term memory (LSTM) architecture of deep neural networks to forecast InSAR-based line-of-sight displacement. The model suitably captures the inherent temporal variations of SAR interferograms. The method implementation on a region near Central and East Java, Indonesia shows satisfactory performance (mean absolute percentage error around 2%) of the proposed LSTM model comprising four layers, each consisting of 100 nodes. We therefore conclude that deep neural network is a promising avenue for InSAR time series analysis.

Modulation Scheme Classification in Cognitive Radio Networks Using the Long Short Term Memory (LSTM) of Deep Learning

Cognitive Radio (CR) system has been adopted for efficient utilization of radio frequency spectrum. The classification of signal modulation schemes is one of the main characteristics of the CR for appropriate demodulation of sensed signals. However, conventional Modulation Classification (MC) techniques require extensive extraction of signal features, which is not often guaranteed. Thus, Deep Learning (DL) has been seen as a promising solution to this drawback in MC. This paper proposes a DL-based MC technique using the Long Short-Term Memory (LSTM) network architecture. The proposed LSTM model was trained on M-ary Phase Shift Keying (MPSK) and M-ary Quadrature Amplitude Modulation (MQAM) signal types. The LSTM directly learns the features of a given modulation scheme of a signal sample during training. The signal samples were generated via computer simulation for Signal-to-Noise Ratios (SNRs) from -10 dB to 20 dB with an interval of 5 dB over flat fading channel and Additive white Gaussian noise (AWGN). Simulation results show that the proposed LSTM model achieves an average classification accuracy of 95% at SNRs above 0 dB.

Deep learning approach to estimate foot pressure distribution in walking with application for a cost-effective insole system

BackgroundFoot pressure distribution can be used as a quantitative parameter for evaluating anatomical deformity of the foot and for diagnosing and treating pathological gait, falling, and pressure sores in diabetes. The objective of this study was to propose a deep learning model that could predict pressure distribution of the whole foot based on information obtained from a small number of pressure sensors in an insole.MethodsTwenty young and twenty older adults walked a straight pathway at a preferred speed with a Pedar-X system in anti-skid socks. A long short-term memory (LSTM) model was used to predict foot pressure distribution. Pressure values of nine major sensors and the remaining 90 sensors in a Pedar-X system were used as input and output for the model, respectively. The performance of the proposed LSTM structure was compared with that of a traditionally used adaptive neuro-fuzzy interference system (ANFIS). A low-cost insole system consisting of a small number of pressure sensors was fabricated. A gait experiment was additionally performed with five young and five older adults, excluding subjects who were used to construct models. The Pedar-X system placed parallelly on top of the insole prototype developed in this study was in anti-skid socks. Sensor values from a low-cost insole prototype were used as input of the LSTM model. The accuracy of the model was evaluated by applying a leave-one-out cross-validation.ResultsCorrelation coefficient and relative root mean square error (RMSE) of the LSTM model were 0.98 (0.92 ~ 0.99) and 7.9 ± 2.3%, respectively, higher than those of the ANFIS model. Additionally, the usefulness of the proposed LSTM model for fabricating a low-cost insole prototype with a small number of sensors was confirmed, showing a correlation coefficient of 0.63 to 0.97 and a relative RMSE of 12.7 ± 7.4%.ConclusionsThis model can be used as an algorithm to develop a low-cost portable smart insole system to monitor age-related physiological and anatomical alterations in foot. This model has the potential to evaluate clinical rehabilitation status of patients with pathological gait, falling, and various foot pathologies when more data of patients with various diseases are accumulated for training.

Classification of imbalanced protein sequences with deep-learning approaches; application on influenza A imbalanced virus classes

Classifiers based on machine learning perform well in the classification of balanced data but struggle with imbalanced data and often merge or ignore the rarer classes, even if the rare classes are more important than other classes. A long-term learning dependency, or Long Short-Term Memory (LSTM) architecture, was developed to compare conventional models with LSTM on polynomial and time-matrix datasets to address the imbalanced classes of influenza virus A. The performances of tree induction and K-Nearest Neighborhood models were less than 90%, and they were not accurate in classifying the classes with fewer samples. The proposed LSTM model can predict all classes reached the highest possible figure of 100%. Thus, for the first time, classification of the imbalanced dataset of influenza virus A at the sequential levels is being reported, which paves the road for the analysis of the proteome-based classification of other proteins.

Splice Junction Identification using Long Short-Term Memory Neural Networks.

BackgroundSplice junctions are the key to move from pre-messenger RNA to mature messenger RNA in many multi-exon genes due to alternative splicing. Since the percentage of multi-exon genes that undergo alternative splicing is very high, identifying splice junctions is an attractive research topic with important implications.ObjectiveThe aim of this paper is to develop a deep learning model capable of identifying splice junctions in RNA sequences using 13,666 unique sequences of primate RNA.MethodsA Long Short-Term Memory (LSTM) Neural Network model is developed that classifies a given sequence as EI (Exon-Intron splice), IE (Intron-Exon splice), or N (No splice). The model is trained with groups of trinucleotides and its performance is tested using validation and test data to prevent bias.ResultsModel performance was measured using accuracy and f-score in test data. The finalized model achieved an average accuracy of 91.34% with an average f-score of 91.36% over 50 runs.ConclusionComparisons show a highly competitive model to recent Convolutional Neural Network structures. The proposed LSTM model achieves the highest accuracy and f-score among published alternative LSTM structures.

A long short-term memory based deep learning algorithm for seismic response uncertainty quantification

The application of metamodeling technique to overcome computational challenge of Monte Carlo simulation (MCS) technique for response uncertainty quantification under stochastic earthquake load is a difficult task due to the high-dimensional nature of stochastic load. Recent developments in the sequential models for forecasting and prediction have opened a new avenue in this regard. Various deep learning algorithms, particularly the convolutional neural network and recurrent neural network are quite suitable for response uncertainty quantification of nonlinear stochastic dynamic system. However, most of the existing studies consider stochastic load as the only source of uncertainty assuming the parameters characterizing a structure as deterministic. The present study proposes a long short-term memory (LSTM) based deep learning algorithm for seismic response uncertainty quantification by duly addressing both the stochastic nature of dynamic load and structural system parameter uncertainty. The functional application program interface feature of Keras that allows layers sharing to form more complex model is explored to form a response approximation model. It incorporates more than one input source i.e., stochastic dynamic excitation sequence as well as structural system parameter uncertainty. The proposed algorithm is elucidated through two numerical examples i.e., a proof-of-concept example and one realistic structural engineering problem by considering the direct MCS based results as the benchmark. The results of accuracy matrices, regression analysis results, comparison of seismic response statistics and reliability results with the results of direct MCS technique clearly revealed the enhanced prediction capability of the proposed LSTM model.

A proactive lane-changing risk prediction framework considering driving intention recognition and different lane-changing patterns

Proactive lane-changing (LC) risk prediction can assist driver’s LC decision-making to ensure driving safety. However, most previous studies on LC risk prediction did not consider the driver’s intention recognition, which made it difficult to guarantee the timeliness and practicability of LC risk prediction. Moreover, the difference in driving risks and its influencing factors between LC to left lane (LCL) and LC to right lane (LCR) have rarely been investigated. To bridge the above research gaps, this study proposes a proactive LC risk prediction framework which integrates the LC intention recognition module and LC risk prediction module. The Long Short-term Memory (LSTM) neural network with time-series input was employed to recognize the driver’s LC intention. The Light Gradient Boosting Machine (LGBM) algorithm was then applied to predict the LC risk. Feature importance analysis was lastly conducted to obtain the key features that affect the LC risk. The highD trajectory dataset was used for framework validation. Results show that the recognition accuracy of the driver’s LCL, LCR and lane-keeping (LK) intentions based on the proposed LSTM model are 97%, 96% and 97%, respectively. Meanwhile, the LGBM algorithm outperforms other machine learning algorithms in LC risk prediction. The results from feature importance analysis show that the interaction characteristics of the LC vehicle and its preceding vehicle in the current lane have the greatest impact on the LC risk. The proposed framework could potentially be implemented in advanced driver-assistance system (ADAS) or autonomous driving system for improved driving safety.

Deep learning model for daily rainfall prediction: case study of Jimma, Ethiopia

Abstract Rainfall prediction is a critical task because many people rely on it, particularly in the agricultural sector. Rainfall forecasting is difficult due to the ever-changing nature of weather conditions. In this study, we carry out a rainfall predictive model for Jimma, a region located in southwestern Oromia, Ethiopia. We propose a Long Short-Term Memory (LSTM)-based prediction model capable of forecasting Jimma's daily rainfall. Experiments were conducted to evaluate the proposed models using various metrics such as Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), Nash–Sutcliffe model efficiency (NSE), and R2, and the results were 0.01, 0.4786, 0.81 and 0.9972, respectively. We also compared the proposed model with existing machine-learning regressions like Multilayer Perceptron (MLP), k-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Decision Tree (DT). The RMSE of MLP was the lowest of the four existing learning models i.e., 0.03. The proposed LSTM model outperforms the existing models, with an RMSE of 0.01. The experimental results show that the proposed model has a lower RMSE and a higher R2.

Recursive long short-term memory network for predicting nonlinear structural seismic response

Artificial neural networks have been used to predict nonlinear structural time histories under seismic excitation because they have a significantly lower computational cost than the traditional time-step integration method. However, most existing techniques require simplification procedures such as downsampling to maintain identical length and sampling rates, and they lack sufficient accuracy, generality, or interpretability. In this paper, a recursive long short-term memory (LSTM) network was proposed for predicting nonlinear structural seismic responses for arbitrary lengths and sampling rates. Referring to the traditional integral solution method, the proposed LSTM model uses the recursive prediction principle and is therefore applicable to structures and earthquakes with different spectral characteristics and amplitudes. The measured ground motions and multilayer frame structures were used for model training and validation. The rules of hyperparameter selection for practical applications are herein discussed. The results showed that the proposed recursive LSTM model can adequately reproduce the global and local characteristics of the time history responses on four different structural response datasets, exhibiting good accuracy and generalization capability.

Time-Lag Selection for Time-Series Forecasting Using Neural Network and Heuristic Algorithm

The time-series forecasting is a vital area that motivates continuous investigate areas of intrigued for different applications. A critical step for the time-series forecasting is the right determination of the number of past observations (lags). This paper investigates the forecasting accuracy based on the selection of an appropriate time-lag value by applying a comparative study between three methods. These methods include a statistical approach using auto correlation function, a well-known machine learning technique namely Long Short-Term Memory (LSTM) along with a heuristic algorithm to optimize the choosing of time-lag value, and a parallel implementation of LSTM that dynamically choose the best prediction based on the optimal time-lag value. The methods were applied to an experimental data set, which consists of five meteorological parameters and aerosol particle number concentration. The performance metrics were: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE) and R-squared. The investigation demonstrated that the proposed LSTM model with heuristic algorithm is the superior method in identifying the best time-lag value.

Energy management optimisation using a combined Long Short-Term Memory recurrent neural network – Particle Swarm Optimisation model

Energy load forecasting is essential to effectively optimise the use of energy for the purpose of smart grid operation. In this work, a focus is placed on Short-Term Load Forecasts (STLF) using measured heat pump data from the UK utilising the Long Short-Term Memory (LSTM) method. A load forecasting program is developed to forecast the electricity consumption of heat pumps based on historical data. LSTM is compared to Backpropagation Neural Network (BPNN), Seasonal Autoregressive Integrated Moving Average (SARIMA), Random Forest (RF) and Holt-Winters Exponential Smoothing (ES). The best average Mean Absolute Percent Error (MAPE) of 1.59% is achieved for the test data set by LSTM, which outperforms other models compared and a number of models presented in papers in the literature. By conducting the Wilcoxon signed-rank test, it is proven that the prediction accuracy improvement of LSTM compared to other models is statistically significant with the exception when comparing with RF with the training data. The impact of data aggregation resolution to forecast accuracy is investigated. The proposed LSTM model performs best on 1-h resolution data compared with other resolution time periods. The results of the LSTM model are integrated with Home Energy Management Systems (HEMS). The electricity costs of the HEMS are compared when using forecasted and measured heat pump data respectively. The results indicate an average percentage difference of 2.06% between the predicted and the actual electricity cost of HEMS for the test data. Furthermore, this paper presents a combined neural network and Particle Swarm Optimisation (PSO) model to develop an optimised energy management system while proving the effectiveness of the LSTM algorithm.

Prediction of the COVID-19 infectivity and the sustainable impact on public health under deep learning algorithm.

The aim is to explore the development trend of COVID-19 (Corona Virus Disease 2019) and predict the infectivity of 2019-nCoV (2019 Novel Coronavirus), as well as its impact on public health. First, the existing data are analyzed through data pre-processing to extract useful feature factors. Then, the LSTM (Long-Short Term Memory) prediction model in the deep learning algorithm is used to predict the epidemic situation in Hubei Province, outside Hubei nationwide, and the whole country, respectively. Meanwhile, the impact of intervention time changes on the epidemic situation is compared. The results show that the prediction results are almost consistent with the actual values. Specifically, Hubei Province abolishes quarantine restrictions after the Spring Festival holiday, and the first COVID-19 peak is reached in late February, while the second COVID-19 peak has been reached in early March. Finally, the cumulative number of diagnoses reaches 85,000 cases, with an increase of 15,000 cases compared with the nationwide cases outside Hubei under the continuous implementation of prevention and control measures. Under the prediction of the proposed LSTM model, if the nationwide implementation of prevention and control interventions is postponed by 5 days, the epidemic will peak in early March, and the cumulative number of diagnoses will be about 200,000; and if the intervention measures are implemented five days earlier, the epidemic will peak in mid-February, with a cumulative number of diagnoses of approximately 40,000. Meanwhile, the proposed LSTM model predicts the RMSE values of the epidemic situation in Hubei Province, outside Hubei nationwide, and the whole country as 34.63, 75.42, and 50.27, respectively. Under model comparison analysis, the prediction error of the proposed LSTM model is small and has better applicability over similar algorithms. The results show that the LSTM model is effective and has high performance in infectious disease prediction, and the research results can provide scientific and effective references for subsequent related research.

Predicting LQ45 financial sector indices using RNN-LSTM

As one of the most popular financial market instruments, the stock has formed one of the most massive and complex financial markets in the world. It could handle millions of transactions within a short period of time and highly unpredictable. In this study, we aim to implement a famous Deep Learning method, namely the long short-term memory (LSTM) networks, for the stock price prediction. We limit the stocks to those that are included in the LQ45 financial sectors indices, i.e., BBCA, BBNI, BBRI, BBTN, BMRI, and BTPS. Rather than using too deep network architecture, we propose using a simple three-layer LSTM network architecture to predict the stocks’ closing prices. We found that the prediction results fall in the reasonable forecasting category. Moreover, it is worth noting that two of the considered stocks, namely, BBCA and BMRI, have the lowest MAPE values at 19.1020 and 18.6135, which fall in the good forecasting results. Hence, the proposed LSTM model is most recommended to be used on those two stocks.

Spindle Thermal Error Prediction Based on LSTM Deep Learning for a CNC Machine Tool

In the precision processing industry, maintaining the accuracy of machine tools for an extensive period is crucial. Machining accuracy is affected by numerous factors, among which spindle thermal elongation caused by an increase in machine temperature is the most common. This paper proposed a key temperature point selection algorithm and thermal error estimation method for spindle displacement in a machine tool. First, highly correlated temperature points were clustered into groups, and the characteristics of small differences within groups and large differences between groups were realized. The optimal number of key temperature points was then determined using the elbow method. Meanwhile, the long short-term memory (LSTM) modeling method was proposed to establish the relationship between the spindle thermal error and changes of the key temperature points. The results show the largest root mean square errors (RMSEs) of the proposed LSTM model and the key temperature point selection algorithm were within 0.6 µm in the spindle thermal displacement experiments with different temperature changes. The results demonstrated that the combined methodology can provide improved accuracy and robustness in predicting the spindle thermal displacement.

Predicting heave and surge motions of a semi-submersible with neural networks

Real-time motion prediction of a vessel or a floating platform can help to improve the performance of motion compensation systems. It can also provide useful early-warning information for offshore operations that are critical with regard to motion. In this study, a long short-term memory (LSTM) -based machine learning model was developed to predict heave and surge motions of a semi-submersible. The training and test data came from a model test carried out in the deep-water ocean basin, at Shanghai Jiao Tong University, China. The motion and measured waves were fed into LSTM cells and then went through several fully connected (FC) layers to obtain the prediction. With the help of measured waves, the prediction extended 46.5 s into future with an average accuracy close to 90%. Using a noise-extended dataset, the trained model effectively worked with a noise level up to 0.8. As a further step, the model could predict motions only based on the motion itself. Based on sensitive studies on the architectures of the model, guidelines for the construction of the machine learning model are proposed. The proposed LSTM model shows a strong ability to predict vessel wave-excited motions.

Novel Multi-Time Scale Deep Learning Algorithm for Solar Irradiance Forecasting

Solar irradiance forecasting is an inevitable and most significant process in grid-connected photovoltaic systems. Solar power is highly non-linear, and thus to manage the grid operation efficiently, with irradiance forecasting for various timescales, such as an hour ahead, a day ahead, and a week ahead, strategies are developed and analysed in this article. However, the single time scale model can perform better for that specific time scale but cannot be employed for other time scale forecasting. Moreover, the data consideration for single time scale forecasting is limited. In this work, a multi-time scale model for solar irradiance forecasting is proposed based on the multi-task learning algorithm. An effective resource sharing scheme between each task is presented. The proposed multi-task learning algorithm is implemented with a long short-term memory (LSTM) neural network model and the performance is investigated for various time scale forecasting. The hyperparameter estimation of the proposed LSTM model is made by a hybrid chicken swarm optimizer based on combining the best features of both the chicken swarm optimization algorithm (CSO) and grey wolf optimization (GWO) algorithm. The proposed model is validated, comparing existing methodologies for single timescale forecasting, and the proposed strategy demonstrated highly consistent performance for all time scale forecasting with improved metric results.

Cellular traffic prediction on blockchain-based mobile networks using LSTM model in 4G LTE network

The various demands from cellular users are increasing day by day. A tower can be selected if the traffic through it at a specific time is predictable. Then a chosen set of issues can also be addressed. In the future, such prediction of traffic will help to identify networks that have better conditions with desired performance. To be specific, in this paper, the Long-Short Term Memory (LSTM) model has been proposed to predict the network traffic at a given 4G LTE cell tower. The LSTM model uses traffic data records of a network for one full year to predict its traffic pattern in a particular week. In addition to this, the 4G LTE access networks are operated via Blockchain. Each LTE tower acts as a separate node like a Peer-to-Peer network. The processes and conditions for accessing mobile networks are coded as smart contracts. Blockchain - Radio Access Network (B-RAN) establishes a secure connection between Access Point (AP) and User Equipment (UE) using smart contracts. If all the conditions of the contract code are satisfied, then data transmission takes place between the user and the AP. The proposed LSTM model on blockchain-enabled network APs to predict LTE data traffic gets validated. It is found that it makes the network encrypted, secure and improves its functioning. The superior performance of the proposed LSTM model justifies that the proposed multilayer LSTM improves the performance from 8.2% to 17.7% as compared to the Autoregressive Integrated Moving Average (ARIMA). The latter is the baseline prediction model.

Concrete compressive strength prediction modeling utilizing deep learning long short-term memory algorithm for a sustainable environment.

One of the most critical parameters in concrete design is compressive strength. As the compressive strength of concrete is correctly measured, time and cost can be decreased. Concrete strength is relatively resilient to impacts on the environment. The production of concrete compressive strength is greatly influenced by severe weather conditions and increases in humidity rates. In this research, a model has been developed to predict concrete compressive strength utilizing a detailed dataset obtained from previously published studies based on a deep learning method, namely, long short-term memory (LSTM), and a conventional machine learning (ML) algorithm, namely, support vector machine (SVM). The input variables of the model include cement, blast furnace slag, fly ash, water, superplasticizer, coarse aggregate, fine aggregate, and age of specimens. To demonstrate the efficiency of the proposed models, three statistical indices, namely, the coefficient of determination (R2), mean absolute error (MAE), and root mean square error (RMSE), were used. Findings shows that LSTM outperformed SVM with R2=0.98, R2= 0.78, MAE=1.861, MAE=6.152, and RMSE=2.36, RMSE=7.93, respectively. The results of this study suggest that high-performance concrete (HPC) compressive strength can be reliably measured using the proposed LSTM model.