Long Short-term Memory Layers Research Articles

DETECTION MODEL FOR FAKE NEWS ON COVID-19 IN INDONESIA

Today, fake information has become a significant problem, exacerbated by the acceleration of access to information. The spread of fake information has a dangerous impact, especially regarding global health issues, for example COVID-19. People can access various resources to obtain information, including online sites and social media. One of the methods to control the spread of false information is detecting hoaxes. Many methods have been developed to identify hoaxes; most previous studies have focused on developing hoax detection methods using data from a single source in English. The present study is carried out to detect fake news in Indonesian language using multiple data sources, including traditional and social media in the context of COVID-19. The study uses Long Short-Term Memory (LSTM) and the Robustly Optimised Bidirectional Encoder Representations from Transformers Pre-Training Approach (RoBERTa). The LSTM approach is used to develop four different architectures that varied based on: (1) the use of text-only versus the use of both title and text; (2) the number of LSTM and dense layers; and (3) the activation function. The LSTM model with text-only data, a single LSTM layer and two dense layers, outperformed other LSTM architectures, achieving the highest accuracy of 92.17%. The LSTM models require a considerably short training time of 23–27 minutes for 3,847 articles and has a detection time of 3.8–4.1 ms per article. The RoBERTa classifiers outperformed all LSTM models with an accuracy of over 97% and a significantly better training time, with a margin of more than 50% compared to LSTM classifiers, although it had a slightly longer test time. Both LSTM and RoBERTa models outperformed the Naïve Bayes and SVM benchmark methods in terms of accuracy, precision, and recall. Therefore, this study shows that both LSTM and RoBERTa methods are reliable and can be reasonably implemented for real-time fake news detection.

Exploring MediaPipe optimization strategies for real-time sign language recognition

The present study meticulously investigates optimization strategies for real-time sign language recognition (SLR) employing the MediaPipe framework. We introduce an innovative multi-modal methodology, amalgamating four distinct Long Short-Term Memory (LSTM) models dedicated to processing skeletal coordinates ascertained from the MediaPipe framework. Rigorous evaluations were executed on esteemed sign language datasets. Empirical findings underscore that the multi-modal approach significantly elevates the accuracy of the SLR model while preserving its real-time capabilities. In comparative analyses with prevalent MediaPipe-based models, our multi-modal strategy consistently manifested superior performance metrics. A distinguishing characteristic of this approach is its inherent adaptability, facilitating modifications within the LSTM layers, rendering it apt for a myriad of challenges and data typologies. Integrating the MediaPipe framework with real-time SLR markedly amplifies recognition precision, signifying a pivotal advancement in the discipline.

LSTM-based energy management algorithm for a vehicle power-split hybrid powertrain

Recurrent neural networks (RNNs) have been used for vehicle speed prediction, trajectory prediction and state diagnosis. As a variant of RNNs, long short-term memory (LSTM) has better state memory ability to deal with problems that have sequential characteristics. In this study, a neural network with two substructures was developed and LSTM was used as the core part for a power-split hybrid powertrain energy management problem, and the optimal control data obtained from the dynamic programming (DP) algorithm was used as the training set. Then on a vehicle model established on coasting-down test data, LSTM conducted real-time control. This paper analyzes the battery status, fuel consumption, carbon dioxide emissions and operating condition of the engine and motor. The results showed that LSTM has good generalization ability and real-time control capability. In the test driving cycles of CHTC-LT and C-WTVC, the theoretical optimal scheme given by the DP algorithm is only 12.15% and 17.96% lower than the LSTM scheme in terms of fuel consumption. In addition, the influence of network size, training epochs and training set on energy-saving effect is compared in detail, it was found that the relatively optimal model required 128 LSTM layer units and 500 training epochs.

Leucogranite mapping via convolutional recurrent neural networks and geochemical survey data in the Himalayan orogen

Geochemical survey data analysis is recognized as an implemented and feasible way for lithological mapping to assist mineral exploration. With respect to available approaches, recent methodological advances have focused on deep learning algorithms which provide access to learn and extract information directly from geochemical survey data through multi-level networks and outputting end-to-end classification. Accordingly, this study developed a lithological mapping framework with the joint application of a convolutional neural network (CNN) and a long short-term memory (LSTM). The CNN-LSTM model is dominant in correlation extraction from CNN layers and coupling interaction learning from LSTM layers. This hybrid approach was demonstrated by mapping leucogranites in the Himalayan orogen based on stream sediment geochemical survey data, where the targeted leucogranite was expected to be potential resources of rare metals such as Li, Be, and W mineralization. Three comparative case studies were carried out from both visual and quantitative perspectives to illustrate the superiority of the proposed model. A guided spatial distribution map of leucogranites in the Himalayan orogen, divided into high-, moderate-, and low-potential areas, was delineated by the success rate curve, which further improves the efficiency for identifying unmapped leucogranites through geological mapping. In light of these results, this study provides an alternative solution for lithologic mapping using geochemical survey data at a regional scale and reduces the risk for decision making associated with mineral exploration.

Prediction of compressive strength in additively fabricated part using long short term memory based neural network

With ability to produce complex geometries and accurate parts, additive manufacturing (AM) could be applied in various sectors including aerospace and biomedical, though the frequent occurrence of defects and unavailability of feasible monitoring and control system prohibits their effective utilization. Technological developments and use of artificial intelligence, machine learning models could be a perfect solution to the problems. In this study, a predictive model is developed using deep learning (DL) algorithm and investigated the compressive strength of fused deposition modelling (FDM) printed part. An improved recurrent neural network (RNN) model called long short-term memory (LSTM) is used for predicting the compressive strength of part produced by FDM process. Model developed with LSTM utilizes layer-wise relevance propagation (LRP) algorithm to establish relation between the FDM part and LSTM layers. Predictive model developed in this study uses the back propagation algorithm to establish the relation between the compressive strength of part and input process variables. A total of 81 poly lactic acid samples were printed and tested, out of which 57 samples were used for training the model and 16 data were used for validating the model and remaining data were used for testing the model. Effects of process variables including infill density, print orientation, raster orientation and layer thickness have been analyzed. The values obtained from the predictive model was in good agreement with experimental results along 1% error. LSTM based DL model predicted the compressive strength closer to the measured values. From the results obtained, the developed model holds good for predicting the compressive strength of FDM printed part.

A CNN and LSTM-based multi-task learning architecture for short and medium-term electricity load forecasting

Electricity load forecasting is the forecast of power load in the future period based on historical load and its related factors. It is of great importance for power system planning, operation, and decision making. There is still an opportunity to improve the precision of forecasting using machine learning and deep learning models, according to previous research findings. In this research a deep learning framework based on multi-task learning (MTL), convolutional neural networks (CNN), and long short-term memory (LSTM) is suggested. By weighing the pertinent training data for the main task and the auxiliary task, the proposed hybrid deep learning network model MTMV-CNN-LSTM addresses the issues of too much repetitive data and poor convolution effect and significantly enhances the generalizability of the model. It uses the CNN layer to extract features from the input data and the LSTM layer for sequence learning. To forecast the short and medium-term electrical load, we created two different trials. The results show that our proposed method is a highly competitive new model with high priority compared to other models for both 10-day short-term and 3-month medium-term power forecasting tasks.

Classification of Epileptic Seizure Types Using Multiscale Convolutional Neural Network and Long Short-Term Memory

The accurate classification of seizure types using electroencephalography (EEG) signals plays a vital role in determining a precise treatment plan and therapy for epilepsy patients. Among the available deep network models, Convolutional Neural Networks (CNNs) are the most widely adopted models for learning and representing EEG signals. However, typical CNNs have high computational complexity, leading to overfitting problems. This paper proposes the design of two effective, lightweight deep network models; the 1D multiscale neural network (1D-MSCNet) model and the Long Short-term Memory (LSTM)-based compact CNN (EEG-LSTMNet) model. The 1D-MSCNet model comprises three modules: a spectral–temporal convolution module, a spatial convolution module, and a classification module. It extracts features from input EEG trials at multiple frequency/time ranges, identifying relationships between the spatial distribution of their channels. The EEG-LSTMNet model includes three convolutional layers, namely temporal, depthwise, and separable layers, a single LSTM layer, and two fully connected classification layers to extract discriminative EEG feature representations. Both models have been applied to the same EEG trials collected from the Temple University Hospital (TUH) database. Results revealed F1-score values of 96.9% and 98.4% for the 1D-MSCNet and EEG-LSTMNet, respectively. Based on the demonstrated outcomes, both models outperform related state-of-the-art methods due to their architectures’ adoption of 1D modules and layers that reduce the computational effort needed, solve the overfitting problem, and enhance classification efficiency. Hence, both models could be valuable additions for neurologists to help them decide upon precise treatments and drugs for patients depending on their type of seizure.

Future degradation trajectory prediction of lithium-ion battery based on a three-step similarity evaluation criterion for battery selection and transfer learning

Accurate remaining useful life (RUL) prediction is important for battery replacement and health maintenance. Due to the complex nonlinear nature of battery degradation, it is difficult to predict the long-term future degradation trend of the battery using its early historical data. Thus, an RUL prediction framework with transfer learning is proposed in this paper. Firstly, considering that the source domain batteries may have different distributions from target batteries due to the inherent inconsistencies between cells, a three-step similarity evaluation criterion is put forward to select similar batteries from the source domain to enhance the ability of positive migration. Secondly, to improve the prediction accuracy, a long short-term memory (LSTM) neural network model with an attention mechanism is developed based on the selected source domain batteries. Finally, a transfer strategy is introduced, the parameters of the LSTM layer and attention layer are shared and those of the rest layers are retrained with limited early historical data of the target battery. The experimental results show that the framework proposed in the paper can effectively predict the future degradation path of the target battery due to full use of the commonality of the batteries degradation and the characteristic of the target battery itself.

DeepCredNet: Empowering Credibility Detection in Online Media Posts with Robust Deep Neural Networks

The rampant dissemination of misinformation and fake news through online media necessitates effective methods for detecting credibility. This study proposes a novel approach that employs a deep neural network (DNN) architecture, specifically Long Short-Term Memory (LSTM), to accurately assess the credibility of online media posts. The proposed LSTM-based DNN model leverages the inherent sequential nature of textual content and metadata associated with the posts. By capturing long-term dependencies and temporal dynamics, the model effectively learns intricate patterns and features crucial for credibility detection. A large labelled dataset comprising both credible and non-credible posts is employed to train the LSTM-based DNN. The input features encompass textual content, user information, and contextual details such as the post source and timing. The LSTM layers within the network enable the model to capture and retain relevant information over extended periods, enhancing its discriminative capabilities. Experimental evaluations validate the efficacy of the proposed approach, showcasing its ability to discern between credible and non-credible online media posts with high accuracy and robust performance. The real-time applicability of this method enables prompt credibility assessment, offering valuable support in combating misinformation and aiding users in making informed decisions while consuming online media.

Vibration-based building health monitoring using spatio-temporal learning model

Vibration-based building health monitoring is a promising and feasible approach to assess the operational state of building structures in a remote, automated, and continuous fashion; however, efficiently handling high-dimensional vibration signals from multiple sensors and effectively coping with missing/noisy data represent two main technical challenges. In order to overcome these issues, this study proposes a novel, reliable and robust framework, abbreviated CLG-BHM, based on a hybrid deep learning architecture. First, the framework uses a 1D convolutional neural network layer to learn low-dimensional representation vectors of long sensor signals, which preserve underlying structures’ dynamic characteristics. Second, temporal relationships within data are distilled via a Long-Short Term Memory layer. Third, the representation vectors of sensors are aggregated with those of their neighbors in a principled way via a graph attention network layer, resulting in a new latent representation rich in both temporal and spatial information. Finally, the latter is gone through a fully-connected layer to provide damage detection results. The performance and viability of the present method are evidenced via various examples involving a simple lumped mass structure, a semi-rigid steel frame, and an experimental 4-story structure from the literature. Moreover, a robustness study is performed, showing that the method can provide reasonable results with the presence of noisy and missing data.

Oil well production prediction based on CNN-LSTM model with self-attention mechanism

To overcome the shortcomings in current study of oil well production prediction, we propose a combined model (CNN-LSTM-SA) with the convolutional neural network (CNN), the long short-term memory (LSTM) neural network and the self-attention mechanism (SA). The CNN-LSTM-SA model consists of five parts: input layer, CNN module, LSTM layer, self-attention layer and output layer. In this model, CNN is used to extract the spatiotemporal features of the input data, LSTM is used to extract the correlation information, and SA is used to capture the internal correlation. Compared with the traditional machine learning methods, such as linear regression (LR), support vector machine (SVM), random forest (RF), XGBoost and back propagation (BP) neural network; and deep learning methods, such as LSTM, LSTM-SA and CNN-LSTM, the CNN-LSTM-SA model can extract the spatial-temporal features that are hidden in oil well production data more comprehensively. It is enable to mine the internal correlation in oil well production data more precisely, thereby improving the accuracy of oil well production prediction. More specifically, among the existing methods, the CNN-LSTM-SA model achieves the best performance in terms of adaptation to the basic trend of oil well production and the prediction of specific values of oil well production.

Denoising odontocete echolocation clicks using a hybrid model with convolutional neural network and long short-term memory network.

Ocean noise negatively influences the recording of odontocete echolocation clicks. In this study, a hybrid model based on the convolutional neural network (CNN) and long short-term memory (LSTM) network-called a hybrid CNN-LSTM model-was proposed to denoise echolocation clicks. To learn the model parameters, the echolocation clicks were partially corrupted by adding ocean noise, and the model was trained to recover the original echolocation clicks. It can be difficult to collect large numbers of echolocation clicks free of ambient sea noise for training networks. Data augmentation and transfer learning were employed to address this problem. Based on Gabor functions, simulated echolocation clicks were generated to pre-train the network models, and the parameters of the networks were then fine-tuned using odontocete echolocation clicks. Finally, the performance of the proposed model was evaluated using synthetic data. The experimental results demonstrated the effectiveness of the proposed model for denoising two typical echolocation clicks-namely, narrowband high-frequency and broadband echolocation clicks. The denoising performance of hybrid models with the different number of convolution and LSTM layers was evaluated. Consequently, hybrid models with one convolutional layer and multiple LSTM layers are recommended, which can be adopted for denoising both types of echolocation clicks.

Hyperparameters analysis of long short-term memory architecture for crop classification

<span lang="EN-US">Deep learning (DL) has seen a massive rise in popularity for remote sensing (RS) based applications over the past few years. However, the performance of DL algorithms is dependent on the optimization of various hyperparameters since the hyperparameters have a huge impact on the performance of deep neural networks. The impact of hyperparameters on the accuracy and reliability of DL models is a significant area for investigation. In this study, the grid Search algorithm is used for hyperparameters optimization of long short-term memory (LSTM) network for the RS-based classification. The hyperparameters considered for this study are, optimizer, activation function, batch size, and the number of LSTM layers. In this study, over 1,000 hyperparameter sets are evaluated and the result of all the sets are analyzed to see the effects of various combinations of hyperparameters as well the individual parameter effect on the performance of the LSTM model. The performance of the LSTM model is evaluated using the performance metric of minimum loss and average loss and it was found that classification can be highly affected by the choice of optimizer; however, other parameters such as the number of LSTM layers have less influence.</span>

Enhancing an Imbalanced Lung Disease X-ray Image Classification with the CNN-LSTM Model

Lung diseases have a significant impact on respiratory health, causing various symptoms and posing challenges in diagnosis and treatment. This research presents a methodology for classifying lung diseases using chest X-ray images, specifically focusing on COVID-19, pneumonia, and normal cases. The study introduces an optimal architecture for convolutional neural network (CNN) and long short-term memory (LSTM) models, considering evaluation metrics and training efficiency. Furthermore, the issue of imbalanced datasets is addressed through the application of some image augmentation techniques to enhance model performance. The most effective model comprises five convolutional blocks, two LSTM layers, and no augmentation, achieving an impressive F1 score of 0.9887 with a training duration of 91 s per epoch. Misclassifications primarily occurred in normal cases, accounting for only 3.05% of COVID-19 data. The pneumonia class demonstrated excellent precision, while the normal class exhibited high recall and an F1 score. Comparatively, the CNN-LSTM model outperformed the CNN model in accurately classifying chest X-ray images and identifying infected lungs. This research provides valuable insights for improving lung disease diagnosis, enabling timely and accurate identification of lung diseases, and ultimately enhancing patients’ outcomes.

Предсказание временных рядов солнечной активности с помощью искусственной нейронной сети LSTM

A numerical model for predicting the parameters of solar activity — the number of sunspots R and the radioflux at a wave of 10.7 cm F10.7 ahead for 27 days — has been built. The numerical model uses an artificial neural network (NN) with LSTM (Long short-term memory) layers. For both the number of sunspots and the radioflux, the model predicts the levels and limits of variation of the values for 27 days. The average absolute prediction error of the model is less than 6 %. The real-time model is implemented on the site http://aurora.pgia.ru and can be an addition to long-term forecasts of other INTERNET resources.

Sentiment Analysis of Comment Data Based on BERT-ETextCNN-ELSTM

With the rapid popularity and continuous development of social networks, users’ communication and interaction through platforms such as microblogs and forums have become more and more frequent. The comment data on these platforms reflect users’ opinions and sentiment tendencies, and sentiment analysis of comment data has become one of the hot spots and difficulties in current research. In this paper, we propose a BERT-ETextCNN-ELSTM (Bidirectional Encoder Representations from Transformers–Enhanced Convolution Neural Networks–Enhanced Long Short-Term Memory) model for sentiment analysis. The model takes text after word embedding and BERT encoder processing and feeds it to an optimized CNN layer for convolutional operations in order to extract local features of the text. The features from the CNN layer are then fed into the LSTM layer for time-series modeling to capture long-term dependencies in the text. The experimental results proved that compared with TextCNN (Convolution Neural Networks), LSTM (Long Short-Term Memory), TextCNN-LSTM (Convolution Neural Networks–Long Short-Term Memory), and BiLSTM-ATT (Bidirectional Long Short-Term Memory Network–Attention), the model proposed in this paper was more effective in sentiment analysis. In the experimental data, the model reached a maximum of 0.89, 0.88, and 0.86 in terms of accuracy, F1 value, and macro-average F1 value, respectively, on both datasets, proving that the model proposed in this paper was more effective in sentiment analysis of comment data. The proposed model achieved better performance in the review sentiment analysis task and significantly outperformed the other comparable models.

Short-term power load forecasting for combined heat and power using CNN-LSTM enhanced by attention mechanism

This study proposes a new approach for short-term power load forecasting using a combination of convolutional neural networks (CNN), long short-term memory (LSTM), and attention mechanisms to address the issue of information loss due to excessively long input time series data. The objective is to enhance the accuracy of short-term power load prediction, which is crucial for efficient energy management. The study analyzes the relationship between the target load and the collected parameters, identifying the most influential factors using Pearson correlation coefficient analysis. A one-dimensional CNN layer is utilized to extract high-dimensional features from the input data, followed by an LSTM layer that captures temporal correlations within the historical sequences. Finally, an attention mechanism is introduced to optimize the weight of the LSTM output, enhance the influence of key information, and optimize the overall prediction model. The performance of the proposed model is evaluated using two benchmark models based on mean absolute percentage error (MAPE), root mean square error (RMSE), and mean absolute error (MAE) metrics. The results show that the CNN-LSTM-A model outperforms the traditional LSTM model regarding power load prediction accuracy for two thermal power units, with an improvement of 7.3% and 5.7%, respectively, indicating superior performance. Therefore, this study demonstrates the effectiveness of the proposed CNN-LSTM-A model for short-term power load forecasting, which has potential applications in the energy industry. In conclusion, the proposed approach can improve the accuracy of power load forecasting, leading to more efficient energy management and cost savings. Additionally, the study highlights the importance of incorporating attention mechanisms into traditional LSTM models for power load forecasting, as it helps to optimize the weight of the LSTM output and improve the accuracy of the predictions. The proposed CNN-LSTM-A model can be potentially useful for energy companies and policymakers in making informed decisions regarding energy production and consumption. Overall, this study provides a valuable contribution to power load forecasting, and the proposed approach could be extended to other areas of time-series forecasting in the future.

A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors.

Gait phase recognition is of great importance in the development of rehabilitation devices. The advantages of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) are combined (LSTM-CNN) in this paper, then a gait phase recognition method based on LSTM-CNN neural network model is proposed. In the LSTM-CNN model, the LSTM layer is used to process temporal sequences and the CNN layer is used to extract features A wireless sensor system including six inertial measurement units (IMU) fixed on the six positions of the lower limbs was developed. The difference in the gait recognition performance of the LSTM-CNN model was estimated using different groups of input data collected by seven different IMU grouping methods. Four phases in a complete gait were considered in this paper including the supporting phase with the right hill strike (SU-RHS), left leg swimming phase (SW-L), the supporting phase with the left hill strike (SU-LHS), and right leg swimming phase (SW-R). The results show that the best performance of the model in gait recognition appeared based on the group of data from all the six IMUs, with the recognition precision and macro-F1 unto 95.03% and 95.29%, respectively. At the same time, the best phase recognition accuracy for SU-RHS and SW-R appeared and up to 96.49% and 95.64%, respectively. The results also showed the best phase recognition accuracy (97.22%) for SW-L was acquired based on the group of data from four IMUs located at the left and right thighs and shanks. Comparably, the best phase recognition accuracy (97.86%) for SU-LHS was acquired based on the group of data from four IMUs located at left and right shanks and feet. Ulteriorly, a novel gait recognition method based on Data Pre-Filtering Long Short-Term Memory and Convolutional Neural Network (DPF-LSTM-CNN) model was proposed and its performance for gait phase recognition was evaluated. The experiment results showed that the recognition accuracy reached 97.21%, which was the highest compared to Deep convolutional neural networks (DCNN) and CNN-LSTM.

Multichannel Approach for Sentiment Analysis Using Stack of Neural Network with Lexicon Based Padding and Attention Mechanism

Abstract Sentiment analysis (SA) has been an important focus of study in the fields of computational linguistics and data analysis for a decade. Recently, promising results have been achieved when applying DNN models to sentiment analysis tasks. Long short-term memory (LSTM) models, as well as its derivatives like gated recurrent unit (GRU), are becoming increasingly popular in neural architecture used for sentiment analysis. Using these models in the feature extraction layer of a DNN results in a high dimensional feature space, despite the fact that the models can handle sequences of arbitrary length. Another problem with these models is that they weight each feature equally. Natural language processing (NLP) makes use of word embeddings created with word2vec. For many NLP jobs, deep neural networks have become the method of choice. Traditional deep networks are not dependable in storing contextual information, so dealing with sequential data like text and sound was a nightmare for such networks. This research proposes multichannel word embedding and employing stack of neural networks with lexicon-based padding and attention mechanism (MCSNNLA) method for SA. Using convolution neural network (CNN), Bi-LSTM, and the attention process in mind, this approach to sentiment analysis is described. One embedding layer, two convolution layers with max-pooling, one LSTM layer, and two fully connected (FC) layers make up the proposed technique, which is tailored for sentence-level SA. To address the shortcomings of prior SA models for product reviews, the MCSNNLA model integrates the aforementioned sentiment lexicon with deep learning technologies. The MCSNNLA model combines the strengths of emotion lexicons with those of deep learning. To begin, the reviews are processed with the sentiment lexicon in order to enhance the sentiment features. The experimental findings show that the model has the potential to greatly improve text SA performance.

A new RNN based machine learning model to forecast COVID-19 incidence, enhanced by the use of mobility data from the bike-sharing service in Madrid

As a respiratory virus, COVID-19 propagates based on human-to-human interactions with positive COVID-19 cases. The temporal evolution of new COVID-19 infections depends on the existing number of COVID-19 infections and the people's mobility. This article proposes a new model to predict upcoming COVID-19 incidence values that combines both current and near-past incidence values together with mobility data. The model is applied to the city of Madrid (Spain). The city is divided into districts. The weekly COVID-19 incidence data per district is used jointly with a mobility estimation based on the number of rides reported by the bike-sharing service in the city of Madrid (BiciMAD). The model employs a Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN) to detect temporal patterns for COVID-19 infections and mobility data, and combines the output of the LSTM layers into a dense layer that can learn the spatial patterns (the spread of the virus between districts). A baseline model that employs a similar RNN but only based on the COVID-19 confirmed cases with no mobility data is presented and used to estimate the model gain when adding mobility data. The results show that using the bike-sharing mobility estimation the proposed model increases the accuracy by 11.7% compared with the baseline model.