Long Short-term Memory Method Research Articles

Predicting PM2.5 concentration with enhanced state-trend awareness and uncertainty analysis using bagging and LSTM neural networks.

Monitoring air pollutants, particularly PM2.5, which refers to fine particulate matter with a diameter of 2.5µm or smaller, has become a focal point of environmental protection efforts worldwide. This study introduces the concept of state-trend awareness, which is widely employed in big data analytics to enhance global threat identification, understanding, and response capabilities. We applied this approach to the prediction of PM2.5, utilizing its capacity to provide holistic insights and support decisions in dynamic environments. We conducted in-depth analyses of extensive historical data to forecast the future concentration trends. By combining a long short-term memory (LSTM) neural network with a bagging ensemble learning algorithm, our developed model demonstrated superior accuracy and generalization compared to those of traditional LSTM and support vector machine (SVM) methods, reducing errors relative to SVM-LSTM by 12%. We further introduced interval prediction to address forecasting uncertainties, not only providing a specific PM2.5 but also forecasting the probability ranges of its variations. The simulation results illustrate the effectiveness of our approach in improving the prediction accuracy, enhancing model generalization, and reducing overfitting, thereby offering a robust analytical tool for environmental monitoring and public health decision-making.

Detection of COVID-19 Anti-Vaccination from Twitter Data Using Deep Learning and Feature Selection Approaches

The COVID-19 pandemic has evolved into a crisis significantly impacting health, the economy, and social life worldwide. During this crisis, anti-vaccination sentiment poses a considerable obstacle to controlling the epidemic and the effectiveness of vaccination campaigns. This study aimed to detect COVID-19 anti-vaccination sentiment from Twitter data using a combination of deep learning and feature selection approaches. The proposed method integrates a deep learning model with feature selection techniques to identify anti-vaccination sentiment by pinpointing important features in text data. Hybrid TF-IDF and N-gram methods were utilized for feature extraction, followed by Chi-square feature selection. The dataset comprises Twitter text data and two labels. The Synthetic Minority Oversampling Technique (SMOTE) was applied to balance the labels. Long Short-Term Memory (LSTM), a deep learning architecture, was employed for the classification process. The experimental results, obtained by leveraging the proposed feature extraction, feature selection, and LSTM methods, achieved the highest accuracy value of 99.23%. These findings demonstrate the proposed methods' success in effectively detecting COVID-19 anti-vaccination sentiment in text data. The study's results can offer valuable insights for developing health policies and public information strategies, presenting a new and powerful tool for detecting anti-vaccine sentiment in planning vaccination campaigns and public health interventions.

Analisis Sentimen Terhadap Kinerja Kepolisian Indonesia Menggunakan Metode Multinomial Naive Bayes, Long Short-Term Memory, dan Lexicon-Based

According to the Indonesian Survey Institute, the public trust index in the Indonesian Police decreased to 53% as of October 2022. The level of public satisfaction and trust in police can be measured by examining how the police handle each case that occurs. This research focuses on analyzing and measuring the public’s sentiment toward the police’s performance among Twitter users in Indonesia. After scraping (collecting) data from Twitter, out of the 14,000 collected tweets, only 12,222 tweets were for analysis after the preprocessing stage. Then, the data was labeled into two categories, negative sentiment and positive sentiment using InSet Lexicon. The results showed that 53% (6,478) tweets have a negative sentiment, while 47% (5,744) tweets have a positive sentiment. The topics frequently appear and discussed related to such as the shooting case of Brigadier J, the Kanjuruhan tragedy, the security of the G20 Bali Summit, securing Christmas 2022 and securing New Year 2023. Subsequently, classification was performed on the test data using the Multinomial Naïve Bayes and Long Short-Term Memory (LSTM) methods. In the testing phase, a confusion matrix used to calculate accuracy, recall, precision, and f1-score. The results showed that the Multinomial Naïve Bayes method achieved an accuracy score of 78%, precision of 78.5%, recall of 77%, and an f1-score of 77%, with an average score of 77.6%. Similarly, the LSTM method resulted in an equal accuracy, precision, recall, and f1-score of 87%.

Machine learning‐based streamflow forecasting using CMIP6 scenarios: Assessing performance and improving hydrological projections and climate change

AbstractWater is essential for humans as well as for all living organisms to sustain their lives. Therefore, any climate‐driven change in available resources has significant impacts on the environment and life. Global climate models (GCMs) are one of the most practical methods to evaluate climate change. Based on this, this research evaluated the capability of GCMs from the Coupled Model Intercomparison Project 6 (CMIP6) to reproduce the historical flow of climate prediction centre data for the Konya Closed basin and to project the climate of the basin using the selected GCMs. Global climate models based on the CMIP6 under the scenario of common socioeconomic pathways (SSP245 and SSP 585) were used to analyse the climate change effect on streamflow of the study area by Bias Correction of GCM Models using Long Short‐Term Memory (LSTM), Bidirectional LSTM (BiLSTM), AdaBoost, Gradient Boosting, Regression Tree, and Random Forest methods. The coefficient of determination (R2), mean square error (MSE), mean absolute error (MAE), root mean square error (RMSE) were used to assess the performance of the methods. Findings show that the Random Forest Model consistently outperformed other models in both the testing and training phases. A significant downward in the volume of water flowing through the region's rivers and streams in the next decades. It is critical to enhance climate‐resilient water infrastructure financing, establish an early warning system for drought, introduce best management practices, implement integrated water resource management, public awareness, and support water research to alleviate the negative consequences of drought and increase resilience against the effects of climate change on Turkey's water resources.

InfusedHeart: A Novel Knowledge-Infused Learning Framework for Diagnosis of Cardiovascular Events

In the undertaken study, we have used a customized dataset termed ``Cardiac-200'' and the benchmark dataset ``PhysioNet.'' which contains 1500 heartbeat acoustic event samples (without augmentation) and 1950 samples (with augmentation) heartbeat acoustic events such as normal, murmur, extrasystole, artifact, and other unlabeled heartbeat acoustic events. The primary reason for designing a customized dataset, ``cardiac-200,'' is to balance the total number of samples into categories such as normal and abnormal heartbeat acoustic events. The average duration of the recorded heartbeat acoustic events is 10-12 s. In the undertaken study, we have analyzed and evaluated various heartbeat acoustic events using audio processing libraries such as Chromagram, Chroma-cq, Chroma-short-time Fourier transform (STFT), Chroma-cqt, and Chroma-cens to extract more information from the recorded heartbeat sound signals. The noise removal process has been carried out using local binary pattern (LBP) methodology. The noise-robust heartbeat acoustic images are classified using long short-term memory (LSTM)-convolutional neural network (CNN), recurrent neural network (RNN), LSTM, Bi-LSTM, CNN, K-means Clustering, and support vector machine (SVM) methods. The obtained results have shown that the proposed InfusedHeart Framework had outclassed all the other customized machine learning and deep learning approaches such as RNN, LSTM, Bi-LSTM, CNN, K-means Clustering, and SVM-based classification methodologies. The proposed Knowledge-infused Learning Framework has achieved an accuracy of 89.36% (without augmentation), 93.38% (with augmentation), and a standard deviation of 10.64 (without augmentation), and 6.62 (with augmentation). Furthermore, the proposed framework has been tested for various signal-to-noise ratio conditions such as SignaltoNoiseRatio0, SignaltoNoiseRatio3, SignaltoNoiseRatio6, SignaltoNoiseRatio9, SignaltoNoiseRatio12, SignaltoNoiseRatio15, and SignaltoNoiseRatio18. In the end, we have shown a detailed comparison of texture and without texture approaches and have discussed future enhancements and prospective ways for future directions.

An Accurate Model for Covid-19 Positive Cases in India by using Traditional ARIMA and Artificial Neural Networks (LSTM and Bi-LSTM)

Objective: This study focuses on evaluating the accuracy of models that can be identified by taking the data of COVID-19-positive cases in India. Methods: To build the models by using the procedures, Artificial Neural Networks (ANN) and Auto Regressive Integrated Moving Average (ARIMA). The data has been taken for various time periods of (Covid-19 positive cases) from March 2022 to July 2023; March 2022 to Nov. 2022 and from Nov. 2022 to July 2023. The data was collected from the official website of the World Health Organization (WHO). The traditional ARIMA and Long Short-Term Memory (LSTM) deep learning methods were applied to build models for various time periods. Findings: Model performance is being measured with the error parameter (Root Mean Square Error) RMSE (215.74, 100.36 and 127.81) respectively for all the time periods. LSTM is performing better than the ARIMA with a minimum value of RMSE. Novelty: The study has been done for the time periods of Covid-19 positive cases with the help of LSTM, Bi-LSTM and ARIMA methods. The outcome of these methods gave; LSTM is the accurate and best performance model. Keywords: ARIMA, Neural Networks, LSTM, RMSE

A cosine adaptive particle swarm optimization based long-short term memory method for urban green area prediction

In the quest for sustainable urban development, precise quantification of urban green space is paramount. This research delineates the implementation of a Cosine Adaptive Particle Swarm Optimization Long Short-Term Memory (CAPSO-LSTM) model, utilizing a comprehensive dataset from Beijing (1998–2021) to train and test the model. The CAPSO-LSTM model, which integrates a cosine adaptive mechanism into particle swarm optimization, advances the optimization of long short-term memory (LSTM) network hyperparameters. Comparative analyses are conducted against conventional LSTM and Partical Swarm Optimization (PSO)-LSTM frameworks, employing mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) as evaluative benchmarks. The findings indicate that the CAPSO-LSTM model exhibits a substantial improvement in prediction accuracy over the LSTM model, manifesting as a 66.33% decrease in MAE, a 73.78% decrease in RMSE, and a 57.14% decrease in MAPE. Similarly, when compared to the PSO-LSTM model, the CAPSO-LSTM model demonstrates a 58.36% decrease in MAE, a 65.39% decrease in RMSE, and a 50% decrease in MAPE. These results underscore the efficacy of the CAPSO-LSTM model in enhancing urban green space area prediction, suggesting its significant potential for aiding urban planning and environmental policy formulation.

Assessment of surface ozone products from downscaled CAMS reanalysis and CAMS daily forecast using urban air quality monitoring stations in Iran

Abstract. Tropospheric ozone time series consist of the effects of various scales of motion, from meso-scales to large timescales, which are often challenging for global models to capture. This study uses two global datasets, namely the reanalysis and the daily forecast of the Copernicus Atmosphere Monitoring Service (CAMS), to assess the capability of these products in presenting ozone's features on regional scales. We obtained 16 relevant meteorological and several pollutant species, such as O3, CO, NOx, etc., from CAMS. Furthermore, we employed a comprehensive set of in situ measurements of ozone at 27 urban stations in Iran for the year 2020. We decomposed the time series into three spectral components, i.e., short (S), medium (M), and long (L) terms. To cope with the scaling issue between the measured data and the CAMS' products, we developed a downscaling approach based on a long short-term memory (LSTM) neural network method which, apart from modeled ozone, also assimilated meteorological quantities as well as lagged O3 observations. Results show the benefit of applying the LSTM method instead of using the original CAMS products for providing O3 over Iran. It is found that lagged O3 observation has a larger contribution than other predictors in improving the LSTM. Compared with the S, the M component shows more associations with observations, e.g., correlation coefficients larger than 0.7 for the S and about 0.95 for the M in both models. The performance of the models varies across cities; for example, the highest error is for areas with high emissions of O3 precursors. The robustness of the results is confirmed by performing an additional downscaling method. This study demonstrates that coarse-scale global model data, such as CAMS, need to be downscaled for regulatory purposes or policy applications at local scales. Our method can be useful not only for the evaluation but also for the prediction of other chemical species, such as aerosols.

Enhancing GNSS Deformation Monitoring Forecasting with a Combined VMD-CNN-LSTM Deep Learning Model

Hydraulic infrastructures are susceptible to deformation over time, necessitating reliable monitoring and prediction methods. In this study, we address this challenge by proposing a novel approach based on the combination of Variational Mode Decomposition (VMD), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM) methods for Global Navigation Satellite Systems (GNSS) deformation monitoring and prediction modeling. The VMD method is utilized to decompose the complex deformation signals into intrinsic mode functions, which are then fed into a CNN method for feature extraction. The extracted features are input into an LSTM method to capture temporal dependencies and make predictions. The experimental results demonstrate that the proposed VMD-CNN-LSTM method exhibits an improvement by about 75%. This research contributes to the advancement of deformation monitoring technologies in water conservancy engineering, offering a promising solution for proactive maintenance and risk mitigation strategies.

Research on self-healing characteristic and state prediction method of the copper based powder metallurgy materials on friction interface

In high power density transmission systems, the friction and wear characteristic of copper based powder metallurgy materials is directly linked to working reliability. Moreover, these materials have frictional self-healing characteristic at the material interface. This paper focuses on exploring the healing mechanism of copper based powder metallurgy materials and conducts ‘damage-healing’ tests, proposing a method to characterize the self-healing characteristic. Subsequently, through comparative tests, the influence of temperature, speed, and pressure on the self-healing characteristics is analyzed. The results show that the increase in temperature reduces the furrow width and depth by 15.30% and 59.76%, respectively. Pressure has the greatest effect on surface roughness, reducing it by 67%. Meanwhile, this paper developed a PSO (Particle Swarm Optimization)-LSTM (Long Short-Term Memory) method to accurately predict the self-healing characterization parameters and self-healing time with small error (average 4.35%) and high correlation coefficient (R 2) (average 0.976). This study contributes to the development of interface repair technology for friction materials.

Classification of Intracranial Hemorrhage (CT) Images Using CNN- LSTM Method and Image-Based GLCM Features

In this study, a hybrid approach is utilized, which combines image-based grey-level co-occurrence matrix attributes and transformed-based attributes. The combination of attribute-based techniques is more accurate in categorizing cerebral hemorrhages in CT images when compared with the transformed attribute-based and image-attribute-based approaches. Deep learning methods, particularly long short-term memory (LSTM), have gained popularity in natural language processing applications like text and sentiment analysis. This paper presents an in-depth deep-learning system that is fully automated. The purpose of the system is to organize radiological data and identify cases of intracranial hemorrhage (ICH) for diagnosis. The structure of the proposed DL model includes 1D convolution neural networks (CNN), long short-term memory (LSTM) units, and a logistic function. A significant dataset consisting of 12,852 radiological reports of head computed tomography (CT) was used to train and assess these components.

Obstructive Sleep Apnea Syndrome Identification Using CNN-LSTM Hybrid Model

Obstructive Sleep Apnea (OSA) is a common sleep problem. It causes breathing issues during sleep. This disturbs oxygen intake and sleep patterns. Detecting OSA early and accurately is important for treatment. In our study, we look into using advanced deep learning for OSA detection. We focus on Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNN), and a new combined method. We also check the impact of different functions and methods on OSA detection accuracy. Our tests reveal some interesting facts. The Nadam optimizer gives the best OSA detection at 95.22% accuracy. The combined CNN and LSTM method achieves 95.38% accuracy. Using LSTM with the sigmoid function, we get a 93.80% accuracy. These results stress the importance of choosing suitable models and methods for OSA detection. Our study helps improve diagnostic tools for OSA. These findings can aid in early and improved treatment of OSA, giving people a better quality of life and health.

BO-STA-LSTM: Building energy prediction based on a Bayesian optimized spatial-temporal attention enhanced LSTM method

In predicting building energy (affected by seasons), there are issues like inefficient hyperparameter optimization and inaccurate predictions, it is unclear whether spatial and temporal attention improves performance. This study proposes a method based on Bayesian Optimization (BO), Spatial-Temporal Attention (STA), and Long Short-Term Memory (LSTM). Seven improved LSTM models (BO-LSTM, SA-LSTM, TA-LSTM, STA-LSTM, BO-SA-LSTM, BO-TA-LSTM, and BO-STA-LSTM) are compared with LSTM and the impacts of seasonal variations on BO-STA-LSTM are analysed using different sample types and time domain analysis. To further demonstrate the efficiency of the proposed method, comparisons with convolutional neural network (CNN) and temporal convolutional network (TCN) are performed, and followed by validation with new datasets. The findings indicate that adding STA and BO to LSTM enhances the average R2 of prediction performance by 0.0885. BO alone contributes 0.0717, while adding attention improves 0.0560. BO-STA-LSTM achieves higher prediction accuracy for similar test and training samples or a test sample size of 14,016, effectively capturing seasonal, trend, and peak energy patterns. Additionally, BO-STA-LSTM outperforms CNN and TCN, demonstrating superior prediction accuracy.

An Integrated Framework with ADD-LSTM and DeepLabCut for Dolphin Behavior Classification

Caring for dolphins is a delicate process that requires experienced caretakers to pay close attention to their behavioral characteristics. However, caretakers may sometimes lack experience or not be able to give their full attention, which can lead to misjudgment or oversight. To address these issues, a dolphin behavior analysis system has been designed to assist caretakers in making accurate assessments. This study utilized image preprocessing techniques to reduce sunlight reflection in the pool and enhance the outline of dolphins, making it easier to analyze their movements. The dolphins were divided into 11 key points using an open-source tool called DeepLabCut, which accurately helped mark various body parts for skeletal detection. The AquaAI Dolphin Decoder (ADD) was then used to analyze six dolphin behaviors. To improve behavior recognition accuracy, the long short-term memory (LSTM) neural network was introduced. The ADD and LSTM models were integrated to form the ADD-LSTM system. Several classification models, including unidirectional and bidirectional LSTM, GRU, and SVM, were compared. The results showed that the ADD module combined with a double-layer bidirectional LSTM method achieved high accuracy in dolphin behavior analysis. The accuracy rates for each behavior exceeded 90%.

Evaluation of LSTM and Wavelet Methods for Wind Speed Forecasting in Bahia, Brazil

The pursuit of sustainable development is intricately linked to the effective management of renewable energy resources, with wind energy emerging as a key player on the global stage. However, the inherent volatility and intermittency of wind patterns pose significant challenges to accurate wind speed forecasting, which is crucial for the stable operation of wind turbines. Our study presents a novel forecasting model that integrates cuttingedge data decomposition techniques with Long Short-Term Memory (LSTM) networks in this context. Our approach leverages advanced methodologies such as Wavelet Transform (WT), Empirical Mode Decomposition (EMD), and Enhanced Empirical Mode Decomposition (EEMD) to segment time series data into distinct high and low-frequency components. These segmented signals are then individually forecasted using Bidirectional LSTM (BiLSTM) networks, with the amalgamation of these predictions providing the final forecast output. Our empirical findings demonstrate that the hybrid model, particularly utilizing EMD and EEMD, exhibits superior performance compared to existing forecasting models in terms of both accuracy and stability. By effectively combining sophisticated data decomposition techniques with state-of-the-art deep learning algorithms, our proposed model offers a robust solution for wind speed forecasting. This facilitates the efficient management of renewable energy resources and advances the cause of sustainable development initiatives worldwide.

Perbandingan Metode Extreme Gradient Boosting (XGBOOST) Dengan Long Short-Term Memory (LSTM) Untuk Prediksi Saham Pt. Bank Mandiri Tbk. (BMRI)

In the continuously evolving world of investments, achieving optimal investment results is the primary goal of every investor. The high profit potential in stock investments makes it an attractive choice. However, it is difficult to predict the direction of stock price movements, but many methods and ways are used to predict in terms of buying and selling, one of which is the result of the very rapid development of computing for machine learning, namely artificial intelligence techniques. In this study, the artificial intelligence methods used are Extreme Gradient Boosting (XGBoost) and Long Short-Term Memory (LSTM). This study uses a stock dataset of PT. Bank Mandiri Tbk. (BMRI) for 20 years, especially in the open price column. After testing, the results from the XGBoost method are obtained, namely the Coefficient of Determination (R2) value of 89.09%, indicating that the results are good and the Mean Absolute Percentage Error (MAPE) is 3.21%, indicating that the error percentage is low. While in the LSTM method, the R2 value is 98.44%, meaning that the prediction results have been predicted very well and the MAPE is 1.77%, indicating that the error percentage is very low. Keywords: XGBoost, Open Price, Prediction, LSTM, Stocks

Sentiment Analysis on the Quality of Public Services with User Satisfaction Prediction of YuhSinau Application Managed by BKPSDM Kabupaten Kebumen Using LSTM Method

The quality of public services is critical in providing effective and responsive governance in an increasingly digital society. The development of the YuhSinau application by the Personnel and Human Resource Development Agency (BKPSDM) of Kebumen Regency offers an innovative response to the growing need for e-learning solutions for local government civil servants (Pegawai Negeri Sipil or PNS). However, determining the app’s effectiveness and user satisfaction is critical. This demands a thorough sentiment analysis in order to acquire insights into users’ thoughts and opinions about the quality of public services supplied by YuhSinau. The Long Short-Term Memory (LSTM) approach is used in this article to examine feelings and forecast customer pleasure. Data collection from multiple sources, initial data preprocessing, LSTM model construction, training, validation, and prediction are all part of the process. The results show that the model has some drawbacks, most notably its failure to appropriately explain variation in target data due to a negative R-squared value. Enhancements to the LSTM architecture, hyperparameter adjustment, and the use of more diverse and representative training data are proposed to improve the model. Continuous review and responsiveness to user comments are critical for improving the quality of public services via the YuhSinau application.
 Keywords: sentiment analysis, public service quality, user satisfaction, YuhSinau application,LSTM method

Comparative Analysis of Commonly Used Machine Learning Approaches for Li-Ion Battery Performance Prediction and Management in Electric Vehicles

The significant role of Li-ion batteries (LIBs) in electric vehicles (EVs) emphasizes their advantages in terms of energy density, being lightweight, and being environmentally sustainable. Despite their obstacles, such as costs, safety concerns, and recycling challenges, LIBs are crucial in terms of the popularity of EVs. The accurate prediction and management of LIBs in EVs are essential, and machine learning-based methods have been explored in order to estimate parameters such as the state of charge (SoC), the state of health (SoH), and the state of power (SoP). Various machine learning techniques, including support vector machines, decision trees, and deep learning, have been employed for predicting LIB states. This study proposes a methodology for comparative analysis, focusing on classical and deep learning approaches, and discusses enhancements to the LSTM (long short-term memory) and Bi-LSTM (bidirectional long short-term memory) methods. Evaluation metrics such as MSE, MAE, RMSE, and R-squared are applied to assess the proposed methods’ performances. The study aims to contribute to technological advancements in the electric vehicle industry by predicting the performance of LIBs. The structure of the rest of the study is outlined, covering materials and methods, LIB data preparation, analysis, the proposal of machine learning models, evaluations, and concluding remarks, with recommendations for future studies.

State of charge estimation for Li-ion battery during dynamic driving process based on dual-channel deep learning methods and conditional judgement

The accurate estimation of the state of charge (SOC) is crucial for the safe and reliable operation of Li-ion batteries. In this paper, novel dual-channel deep learning methods were proposed for SOC estimation during dynamic driving cycles. For dual-channel deep learning methods, each channel of the deep learning methods could be a single convolutional neural network (CNN) layer, single long short-term memory (LSTM) layer or sequentially combined structure of CNN and LSTM. After evaluation, it could be found that dual-channel deep learning methods could achieve higher accuracy than conventional single-channel method, however, the model size and floating-point operations (FLOPs) could be either reduced or increased. Among all the dual-channel deep learning methods, CNN + LSTM-CNN + LSTM method could achieve the highest accuracy and the shortest training time with sacrifices in computational complexity. CNN–CNN + LSTM method could achieve higher accuracy than the single-channel deep learning method with reduced computational complexity and model size. As the error of SOC estimation is highly correlated with voltage signals, conditional judgement was then integrated with dual-channel deep learning method to obtain higher accuracy. After integrated with conditional judgement, the averaged error of SOC estimation could be reduced by more than 45% without a substantial sacrifice in computational time.

A Combined Method for Short-Term Load Forecasting Considering the Characteristics of Components of Seasonal and Trend Decomposition Using Local Regression

In response to the complexity and high volatility of original load data affecting the accuracy of load forecasting, a combined method for short-term load forecasting considering the characteristics of components of seasonal and trend decomposition using local regression (STL) is proposed. The original load data are decomposed into a trend component, seasonal component, and residual component using STL. Then, considering the characteristics of each component, a long short-term memory (LSTM) neural network, a convolutional neural network (CNN), and Gaussian process regression (GPR) are used to predict the trend component, seasonal component, and residual component, respectively. The final outcome of the load forecasting is obtained by summing the forecasted results of each individual component. A specific case study is conducted to compare the proposed combined method with LSTM, CNN, GPR, STL-LSTM, STL-CNN, and STL-GPR prediction methods. Through comparison, the proposed combined method exhibits lower errors and higher accuracy, demonstrating the effectiveness of this method.