Deep Long Short-term Memory Research Articles

Deep Sentiment Analysis of Twitter Data Using a Hybrid Ghost Convolution Neural Network Model.

Several problems remain, despite the evident advantages of sentiment analysis of public opinion represented on Twitter and Facebook. On complicated training data, hybrid approaches may reduce sentiment mistakes. This research assesses the dependability of numerous hybrid approaches on a variety of datasets. Across domains and datasets, we compare hybrid models to singles. Text tweets and reviews are included in our deep sentiment analysis learning systems. The support vector machine (SVM), Long Short-Term Memory (LSTM), and ghost model convolution neural network (CNN) are combined to get the hybrid model. The dependability and computation time of each approach were evaluated. On all datasets, hybrid models outperform single models when deep learning and SVM are combined. The traditional models were less trustworthy, and deep learning algorithms have recently shown their enormous promise in sentiment analysis. Linear transformations are used in feature maps to eliminate duplicate or related features. The ghost unit makes ghost features by taking away attributes that are both similar and duplicated from each intrinsic feature. LSTM produces higher results but takes longer to process, while CNN needs less hyperparameter adjusting and monitoring. The effectiveness of the integrated model varies depending on the work, and all performed better than the others. For hybrid deep sentiment analysis learning models, LSTM networks, CNNs, and SVMs are needed. Hybrid models are used to compare SVM, LSTM, and CNN, and we tested each method's accuracy and errors. Deep learning-SVM hybrid models improve sentiment analysis accuracy. Experimental results have shown the accuracy of the proposed model shown 91.3 percent and 91.5 percent for datasets type 1 and 8, respectively.

Comparative Evaluation and Comprehensive Analysis of Machine Learning Models for Regression Problems

Abstract Artificial intelligence and machine learning applications are of significant importance almost in every field of human life to solve problems or support human experts. However, the determination of the machine learning model to achieve a superior result for a particular problem within the wide real-life application areas is still a challenging task for researchers. The success of a model could be affected by several factors such as dataset characteristics, training strategy and model responses. Therefore, a comprehensive analysis is required to determine model ability and the efficiency of the considered strategies. This study implemented ten benchmark machine learning models on seventeen varied datasets. Experiments are performed using four different training strategies 60:40, 70:30, and 80:20 hold-out and five-fold cross-validation techniques. We used three evaluation metrics to evaluate the experimental results: mean squared error, mean absolute error, and coefficient of determination (R2 score). The considered models are analyzed, and each model's advantages, disadvantages, and data dependencies are indicated. As a result of performed excess number of experiments, the deep Long-Short Term Memory (LSTM) neural network outperformed other considered models, namely, decision tree, linear regression, support vector regression with a linear and radial basis function kernels, random forest, gradient boosting, extreme gradient boosting, shallow neural network, and deep neural network. It has also been shown that cross-validation has a tremendous impact on the results of the experiments and should be considered for the model evaluation in regression studies where data mining or selection is not performed.

Image Captioning Generator Using CNN and LSTM

Abstract: The project's goal is to come up with a caption for an image. Photograph captioning is the process of making a description for an image. It necessitates an understanding of the important things, their characteristics, and the relationships between them.an image's objects Deep learning techniques have progressed, and as a result, we can create models that can predict the future thanks to the availability of large datasets and computing power. Create captions for a picture This is what we've done in Python-based research in which we applied CNN's deep learning algorithm (Convolutional Neural Networks) and LSTM (Long Short-Term Memory) are two types of neural networks. Combining different types of RNNs (Recurrent Neural Networks) so that computer vision can be used A computer can recognize the context of a picture and present it in the appropriate context

A novel self-supervised deep LSTM network for industrial temperature prediction in aluminum processes application

This article studies the influence of pot temperature or electrolyte temperature in the aluminum reduction production. Specifically, these indexes reflect the distribution of the physical and energy field of the reduction cell, the current efficiency, and the lifespan of the aluminum reduction cell. Therefore, the pot temperature detection and identification are two critical and significant issues in the whole production process of aluminum electrolysis. However, due to the low measurement accuracy and high maintenance costs with the thermocouple sensor in the practical production process, the real-time measurement of pot temperature index is still a major challenge, which motivate us to develop a self-supervised soft sensor method based on deep long-short term memory (LSTM). Under the constraint of the limited samples, the proposed method achieves a competitive performance. First, the input variables are selected according to the expert experience. Then, a deep self-supervised model is built. Finally, the proposed self-supervised LSTM model is applied to real-time detection in an industrial electrolysis production case. The performance in the experiment outperforms other existing methods in terms of both accuracy and robustness aspects.

A Compressor Off-Line Washing Schedule Optimization Method With a LSTM Deep Learning Model Predicting the Fouling Trend

Abstract Compressor fouling is one of the most prevalent fault modes that contribute to the performance degradation of a gas turbine power plant. Off-line washing is a standard maintenance procedure to recover the fouling degradation, but with washing cost. In this paper, an off-line washing schedule optimization method with a long short-term memory (LSTM) prediction model is proposed to maximize the plant net profit. First, a mechanism model-based gas path analysis method is developed to identify the fouling indications of compressor flow rate degradation (DGC) and compressor efficiency degradation (DEC). Second, a sliding window prediction method based on LSTM is proposed to accurately predict the nonlinear fouling trends. The prediction models are trained and tested by the true trends of the DGC and DEC that are identified from the field data of a real gas turbine power plant. The comparison results prove that the LSTM algorithm outperforms other machine learning algorithms. The mean relative square error of the DGC LSTM model is 9.72 × 10−4, and DEC is 4.08 × 10−4. Finally, a detailed economic model is developed by coupling the fouling prediction model with the gas turbine performance model. On this basis, an optimization method of the washing schedule is developed to maximize the net profit. Two case studies, under full load and field data, are carried out to verify the proposed optimization method. The results show that the washing schedules of the two case studies are much similar, in which three washing tasks with gradually reduced intervals are provided. Furthermore, the comparison results of different schedules show that the proposed optimal schedule has a huge potential in saving the net profit. It can save 3.26 million Yuan compared with the practical schedule adopted by the real power plant.

Smart Healthcare: Disease Prediction Using the Cuckoo-Enabled Deep Classifier in IoT Framework

The Internet of Things (IoT) is commonly employed to detect different kinds of diseases in the health sector. Presently, disease detection is performed using MRI images, X-rays, CT scans, and so on for diagnosing the diseases. The manual detection process is found to be time-consuming and may result in detection errors that affect the diagnosis. Hence, there is a need for an automatic system for which the deep learning methods gain a major interest. Hence, the idea to combine deep learning and disease prediction to effectively predict the disease is initiated. In this research, the deep learning method is combined with deep learning for the effective prediction of diseases, where the IoT network is employed in the data collection from the patients. The proposed cuckoo-based deep convolutional long-short term memory (deep convLSTM) classifier is employed for disease prediction, where the cuckoo search optimization is utilized for tuning the deep convLSTM classifier. The proposed method is compared with the conventional methods, and it achieved a training percentage of 97.591%, 95.874%, and 97.094%, respectively, for accuracy, sensitivity, and specificity. The comparative analysis proved that the proposed method obtained higher accuracy than other methods.

Deep learning LSTM for predicting thermally induced geometric errors using rotary axes’ powers as input parameters

Temperature changes within a machine tool, which are partly due to internal heat sources related to the power consumption at the machine drives, result in thermally induced geometric errors of the machine. Predicting the thermally affected machine geometry facilitates the prediction of volumetric errors throughout the 5-axis machine volume. Measuring power instead of temperature is easier to implement in practice but causes drift issues. Stacked Long Short Term Memory (LSTM) with two hidden LSTM layers is applied to determine the relationship between powers and the thermal variation of geometric error parameters by remembering previous states and learning complex non-linear relationships through optimized generalized parameters of the predicting model. Optimizers Adam, RMSprop, and SGD based on exponential moving average of first order and second order moments of gradients are combined with Stacked LSTM for faster learning convergence to an optimal solution. Validation trials with various B- and C-axis motion sequences demonstrate a capability of predicting multi time steps ahead over a 40 hours period without re-measuring the geometric error parameters of the model. Stacked LSTM with the Adam optimizer has the best capability by predicting up to 93% of the main geometric error compared to the other optimizers.

Interpretable deep learning LSTM model for intelligent economic decision-making

For sustainable economic growth, information about economic activities and prospects is critical to decision-makers such as governments, central banks, and financial markets. However, accurate predictions have been challenging due to the complexity and uncertainty of financial and economic systems amid repeated changes in economic environments. This study provides two approaches for better economic prediction and decision-making. We present a deep learning model based on the long short-term memory (LSTM) network architecture to predict economic growth rates and crises by capturing sequential dependencies within the economic cycle. In addition, we provide an interpretable machine learning model that derives economic patterns of growth and crisis through efficient use of the eXplainable AI (XAI) framework. For major G20 countries from 1990 to 2019, our LSTM model outperformed other traditional predictive models, especially in emerging countries. Moreover, in our model, private debt in developed economies and government debt in emerging economies emerged as major factors that limit future economic growth. Regarding the economic impact of COVID-19, we found that sharply reduced interest rates and expansion of government debt increased the probability of a crisis in some emerging economies in the future.

Hybrid approach with Deep Auto-Encoder and optimized LSTM based Deep Learning approach to detect anomaly in cloud logs

Anomaly-based detection is coupled with recognizing the uncommon, to catch the unusual activity, and to find the strange action behind that activity. Anomaly-based detection has a wide scope of critical applications, from bank application security to regular sciences to medical systems to marketing apps. Anomaly-based detection adopted by various Machine Learning techniques is really a type of system that consists of artificial intelligence. With the ever-expanding volume and new sorts of information, for example, sensor information from an incontestably enormous amount of IoT devices and from network flow data from cloud computing, it is implicitly understood without surprise that there is a developing enthusiasm for having the option to deal with more conclusions automatically by means of AI and ML applications. But with respect to anomaly detection, many applications of the scheme are simply the passion for detection. In this paper, Machine Learning (ML) techniques, namely the SVM, Isolation forest classifiers experimented and with reference to Deep Learning (DL) techniques, the proposed DA-LSTM (Deep Auto-Encoder LSTM) model are adopted for preprocessing of log data and anomaly-based detection to get better performance measures of detection. An enhanced LSTM (long-short-term memory) model, optimizing for the suitable parameter using a genetic algorithm (GA), is utilized to recognize better the anomaly from the log data that is filtered, adopting a Deep Auto-Encoder (DA). The Deep Neural network models are utilized to change over unstructured log information to training ready features, which are reasonable for log classification in detecting anomalies. These models are assessed, utilizing two benchmark datasets, the Openstack logs, and CIDDS-001 intrusion detection OpenStack server dataset. The outcomes acquired show that the DA-LSTM model performs better than other notable ML techniques. We further investigated the performance metrics of the ML and DL models through the well-known indicator measurements, specifically, the F-measure, Accuracy, Recall, and Precision. The exploratory conclusion shows that the Isolation Forest, and Support vector machine classifiers perform roughly 81%and 79%accuracy with respect to the performance metrics measurement on the CIDDS-001 OpenStack server dataset while the proposed DA-LSTM classifier performs around 99.1%of improved accuracy than the familiar ML algorithms. Further, the DA-LSTM outcomes on the OpenStack log data-sets show better anomaly detection compared with other notable machine learning models.

Making the hospital smart: using a deep long short-term memory model to predict hospital performance metrics

PurposeAbundant studies of outpatient visits apply traditional recurrent neural network (RNN) approaches; more recent methods, such as the deep long short-term memory (DLSTM) model, have yet to be implemented in efforts to forecast key hospital data. Therefore, the current study aims to reports on an application of the DLSTM model to forecast multiple streams of healthcare data.Design/methodology/approachAs the most advanced machine learning (ML) method, static and dynamic DLSTM models aim to forecast time-series data, such as daily patient visits. With a comparative analysis conducted in a high-level, urban Chinese hospital, this study tests the proposed DLSTM model against several widely used time-series analyses as reference models.FindingsThe empirical results show that the static DLSTM approach outperforms seasonal autoregressive integrated moving averages (SARIMA), single and multiple RNN, deep gated recurrent units (DGRU), traditional long short-term memory (LSTM) and dynamic DLSTM, with smaller mean absolute, root mean square, mean absolute percentage and root mean square percentage errors (RMSPE). In particular, static DLSTM outperforms all other models for predicting daily patient visits, the number of daily medical examinations and prescriptions.Practical implicationsWith these results, hospitals can achieve more precise predictions of outpatient visits, medical examinations and prescriptions, which can inform hospitals' construction plans and increase the efficiency with which the hospitals manage relevant information.Originality/valueTo address a persistent gap in smart hospital and ML literature, this study offers evidence of the best forecasting models with a comparative analysis. The study extends predictive methods for forecasting patient visits, medical examinations and prescriptions and advances insights into smart hospitals by testing a state-of-the-art, deep learning neural network method.

Channel based epilepsy seizure type detection from electroencephalography (EEG) signals with machine learning techniques

Epileptic seizures result from disturbances in the electrical activity of the brain, classified as focal, generalized, or unknown. Failure to correctly classify epileptic seizures may result in inappropriate treatment and continuation of seizures. Therefore, automatic detection of generalized, focal, and other epileptic seizures from EEG signals is important. In this research article, Focal-Generalized classification method is proposed that compares traditional classification algorithms and deep learning methods. Two different classifications: four-class (Case (I) Complex Partial Seizure (CPSZ) (C4-T4 Onset)-CPSZ (FP2-F8 Onset)-CPSZ (T5-O1 Onset)- Absence Seizure (ABSZ)) and two-class (Case (II) CPSZ-ABSZ) problems are considered. This study includes preprocessing of scalp Electroencephalogram (EEG) data, feature extraction with discrete wavelet method, feature selection using Correlation-based Feature Selection (CFS) method, and classification of data with classifier algorithms (K-Nearest Neighbors (Knn), Support Vector Machine (SVM), Random Forest (RF) and Long Short-Term Memory (LSTM). The proposed method was applied on 23 subjects in the Temple University Hospital (TUH) scalp EEG data set, and a classification success rate of 95,92% for case (I) and 98,08% for case (II) was successfully achieved with deep learning architecture LSTM.

Spontaneous Speech Emotion Recognition Using Multiscale Deep Convolutional LSTM

Recently, emotion recognition in real sceneries such as in the wild has attracted extensive attention in affective computing, because existing spontaneous emotions in real sceneries are more challenging and difficult to identify than other emotions. Motivated by the diverse effects of different lengths of audio spectrograms on emotion identification, this paper proposes a multiscale deep convolutional long short-term memory (LSTM) framework for spontaneous speech emotion recognition. Initially, a deep convolutional neural network (CNN) model is used to learn deep segment-level features on the basis of the created image-like three channels of spectrograms. Then, a deep LSTM model is adopted on the basis of the learned segment-level CNN features to capture the temporal dependency among all divided segments in an utterance for utterance-level emotion recognition. Finally, different emotion recognition results, obtained by combining CNN with LSTM at multiple lengths of segment-level spectrograms, are integrated by using a score-level fusion strategy. Experimental results on two challenging spontaneous emotional datasets, i.e., the AFEW5.0 and BAUM-1s databases, demonstrate the promising performance of the proposed method, outperforming state-of-the-art methods.

Research on Urban Economic High-Performance Forecasting Method Based on Deep Confidence Network

To address the complexity of urban economic data and the problem that traditional forecasting methods do not fully utilize the correlation of data, resulting in low prediction accuracy, an urban economic forecasting model based on the fusion of deep belief neural (DBN) and long-short term memory (LSTM) is proposed. The model is based on a combination of DBN and LSTM. The model first uses bandpass filtering to denoise the urban economic data and then determines the prediction starting point of the model based on the root-mean-square and cliff features in the trend diagram of the urban economy; secondly, the optimised 4-layer DBN network is used for deep feature extraction and training and testing of the LSTM. The reliability of the proposed model is demonstrated through urban economic experiments, and the prediction results are compared with those of traditional LSTM, BP (back propagation) neural network, and DBN-BP model to verify the effectiveness of the model.

A novel rainfall prediction model based on CEEMDAN-PSO-ELM coupled model

Abstract Rainfall prediction is a very important guideline for water resources management as well as ecological protection, and its changes are the result of multiple factors with obvious uncertainties and nonlinearities. Based on the advantages of Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) non-smooth signal decomposition, the Particle Swarm Optimization (PSO) can be used to optimize the input weights and thresholds of the Extreme Learning Machine (ELM), which can effectively improve the prediction effect and accuracy of ELM, and a rainfall prediction model based on CEEMDAN-PSO-ELM is constructed. The model is applied to the monthly rainfall prediction of Zhongwei City, and the results show that the CEEMDAN-PSO-ELM coupled model has a high prediction accuracy, the mean absolute error (MAE) is 1.29, relative percentage error (RPE) is 0.45, root mean square error (RMSE) is 1.43 and the Nash efficiency coefficient (NSE) is 0.93. It has obvious advantages in hydrological simulation prediction when compared and analyzed with the deep Long-Short Term Memory (LSTM), PSO-ELM coupled model and ELM model.

When machine learning meets fractional-order chaotic signals: detecting dynamical variations

The challenge of classifying multivariate time series generated by discrete and continuous dynamical systems according to their chaotic or non-chaotic behavior has been studied extensively in the literature. The examination of noise or the variation of variables that affect a dynamic system's chaoticity will not be beneficial in analyzing structures employing random number generators (RNG) that are already assured to be chaotic. However, detecting the structural changes and their time intervals in deterministic systems with proven chaoticity can contribute to the literature in encryption applications. Machine Learning algorithms provide flexible possibilities to analyze and predict manipulations that may occur in the dynamics of chaotic and complex systems. This study proposes a deep Long-Short-Term-Memory (LSTM) network with a classification process to predict dynamical changes in a fractional-order chaotic (FOC) system. First, the appropriate system parameters are calculated to satisfy the chaotic behavior in the fractional-order Chen system. The predictive-corrective Adams-Bashforth-Moulton algorithm is used to simulate the FOC Chen system in the time domain. The Lyapunov exponents of the system were obtained according to the Wolf method. Next, three different scenarios have been designed to test and demonstrate the effectiveness of the proposed method. Synthetic FOC signals obtained after sub-sampling and statistical feature extraction processes fed the input of the deep bidirectional LSTM (BiLSTM) network to perform the training and testing process. The classification performance for "q" and "c" classes reaches 100% with the proposed model. The overall average testing accuracy, sensitivity, specificity, precision, F1 score and MCC are 98%, 98%, 99.3%, 98.1%, 98%, and 97.3%, respectively. Our results demonstrate the utility of using a deep BiLSTM network for detecting dynamical variations in nonlinear FOC systems.

A deep LSTM network for the Spanish electricity consumption forecasting

Nowadays, electricity is a basic commodity necessary for the well-being of any modern society. Due to the growth in electricity consumption in recent years, mainly in large cities, electricity forecasting is key to the management of an efficient, sustainable and safe smart grid for the consumer. In this work, a deep neural network is proposed to address the electricity consumption forecasting in the short-term, namely, a long short-term memory (LSTM) network due to its ability to deal with sequential data such as time-series data. First, the optimal values for certain hyper-parameters have been obtained by a random search and a metaheuristic, called coronavirus optimization algorithm (CVOA), based on the propagation of the SARS-Cov-2 virus. Then, the optimal LSTM has been applied to predict the electricity demand with 4-h forecast horizon. Results using Spanish electricity data during nine years and half measured with 10-min frequency are presented and discussed. Finally, the performance of the proposed LSTM using random search and the LSTM using CVOA is compared, on the one hand, with that of recently published deep neural networks (such as a deep feed-forward neural network optimized with a grid search) and temporal fusion transformers optimized with a sampling algorithm, and, on the other hand, with traditional machine learning techniques, such as a linear regression, decision trees and tree-based ensemble techniques (gradient-boosted trees and random forest), achieving the smallest prediction error below 1.5%.

Multi-step-ahead prediction of river flow using NARX neural networks and deep learning LSTM

Abstract Kelantan river (Sungai Kelantan in Malaysia) basin is one of the essential catchments as it has a history of flood events. Numerous studies have been conducted in river basin modelling for the prediction of flow and mitigation of flooding events as well as water resource management. Therefore, having multi-step-ahead forecasting for river flow (RF) is of important research interest in this regard. This study presents four different approaches for multi-step-ahead forecasting for the Kelantan RF, using NARX (nonlinear autoregressive with exogenous inputs) neural networks and deep learning recurrent neural networks called LSTM (long short-term memory). The dataset used was obtained in monthly record for 29 years between January 1988 and December 2016. The results show that two recursive methods using NARX and LSTM are able to do multi-step-ahead forecasting on 52 series of test datasets with NSE (Nash–Sutcliffe efficiency coefficient) values of 0.44 and 0.59 for NARX and LSTM, respectively. For few-step-ahead forecasting, LSTM with direct sequence-to-sequence produces promising results with a good NSE value of 0.75 (in case of two-step-ahead forecasting). However, it needs a larger data size to have better performance in longer-step-ahead forecasting. Compared with other studies, the data used in this study is much smaller.

Task Offloading Based on LSTM Prediction and Deep Reinforcement Learning for Efficient Edge Computing in IoT

In IoT (Internet of Things) edge computing, task offloading can lead to additional transmission delays and transmission energy consumption. To reduce the cost of resources required for task offloading and improve the utilization of server resources, in this paper, we model the task offloading problem as a joint decision making problem for cost minimization, which integrates the processing latency, processing energy consumption, and the task throw rate of latency-sensitive tasks. The Online Predictive Offloading (OPO) algorithm based on Deep Reinforcement Learning (DRL) and Long Short-Term Memory (LSTM) networks is proposed to solve the above task offloading decision problem. In the training phase of the model, this algorithm predicts the load of the edge server in real-time with the LSTM algorithm, which effectively improves the convergence accuracy and convergence speed of the DRL algorithm in the offloading process. In the testing phase, the LSTM network is used to predict the characteristics of the next task, and then the computational resources are allocated for the task in advance by the DRL decision model, thus further reducing the response delay of the task and enhancing the offloading performance of the system. The experimental evaluation shows that this algorithm can effectively reduce the average latency by 6.25%, the offloading cost by 25.6%, and the task throw rate by 31.7%.

Comparative analysis of Gated Recurrent Units (GRU), long Short-Term memory (LSTM) cells, autoregressive Integrated moving average (ARIMA), seasonal autoregressive Integrated moving average (SARIMA) for forecasting COVID-19 trends

Several machine learning and deep learning models were reported in the literature to forecast COVID-19 but there is no comprehensive report on the comparison between statistical models and deep learning models. The present work reports a comparative time-series analysis of deep learning techniques (Recurrent Neural Networks with GRU and LSTM cells) and statistical techniques (ARIMA and SARIMA) to forecast the country-wise cumulative confirmed, recovered, and deaths. The Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) cells based on Recurrent Neural Networks (RNN), ARIMA and SARIMA models were trained, tested, and optimized to forecast the trends of the COVID-19. We deployed python to optimize the parameters of ARIMA which include (p, d, q) representing autoregressive and moving average terms and parameters of SARIMA model include additional seasonal terms which are denoted by (P, D, Q). Similarly, for LSTM and GRU based RNN models’ parameters (number of layers, hidden size, learning rate and number of epochs) are optimized by deploying PyTorch machine learning framework. The best model was chosen based on the lowest Mean Square Error (MSE) and Root Mean Squared Error (RMSE) values. For most of the time-series data of the countries, deep learning-based models LSTM and GRU outperformed statistical ARIMA and SARIMA models, with an RMSE values that are 40 folds less than that of the ARIMA models. But for some countries statistical (ARIMA, SARIMA) models outperformed deep learning models. Further, we emphasize the importance of various factors such as age, preventive measures and healthcare facilities etc. that play vital role on the rapid spread of COVID-19 pandemic.

Live Streaming Speech Recognition Using Deep Bidirectional LSTM Acoustic Models and Interpolated Language Models

Although Long-Short Term Memory (LSTM) networks and deep Transformers are now extensively used in offline ASR, it is unclear how best offline systems can be adapted to work with them under the streaming setup. After gaining considerable experience on this regard in recent years, in this paper we show how an optimized, low-latency streaming decoder can be built in which bidirectional LSTM acoustic models, together with general interpolated language models, can be nicely integrated with minimal perfomance degradation. In brief, our streaming decoder consists of a one-pass, real-time search engine relying on a limited-duration window sliding over time and a number of ad hoc acoustic and language model pruning techniques. Extensive empirical assessment is provided on truly streaming tasks derived from the well-known LibriSpeech and TED talks datasets, as well as from TV shows on a main Spanish broadcasting station.