Long Short-term Memory Method Research Articles

A cosine adaptive particle swarm optimization based long-short term memory method for urban green area prediction

In the quest for sustainable urban development, precise quantification of urban green space is paramount. This research delineates the implementation of a Cosine Adaptive Particle Swarm Optimization Long Short-Term Memory (CAPSO-LSTM) model, utilizing a comprehensive dataset from Beijing (1998–2021) to train and test the model. The CAPSO-LSTM model, which integrates a cosine adaptive mechanism into particle swarm optimization, advances the optimization of long short-term memory (LSTM) network hyperparameters. Comparative analyses are conducted against conventional LSTM and Partical Swarm Optimization (PSO)-LSTM frameworks, employing mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) as evaluative benchmarks. The findings indicate that the CAPSO-LSTM model exhibits a substantial improvement in prediction accuracy over the LSTM model, manifesting as a 66.33% decrease in MAE, a 73.78% decrease in RMSE, and a 57.14% decrease in MAPE. Similarly, when compared to the PSO-LSTM model, the CAPSO-LSTM model demonstrates a 58.36% decrease in MAE, a 65.39% decrease in RMSE, and a 50% decrease in MAPE. These results underscore the efficacy of the CAPSO-LSTM model in enhancing urban green space area prediction, suggesting its significant potential for aiding urban planning and environmental policy formulation.

Assessment of surface ozone products from downscaled CAMS reanalysis and CAMS daily forecast using urban air quality monitoring stations in Iran

Abstract. Tropospheric ozone time series consist of the effects of various scales of motion, from meso-scales to large timescales, which are often challenging for global models to capture. This study uses two global datasets, namely the reanalysis and the daily forecast of the Copernicus Atmosphere Monitoring Service (CAMS), to assess the capability of these products in presenting ozone's features on regional scales. We obtained 16 relevant meteorological and several pollutant species, such as O3, CO, NOx, etc., from CAMS. Furthermore, we employed a comprehensive set of in situ measurements of ozone at 27 urban stations in Iran for the year 2020. We decomposed the time series into three spectral components, i.e., short (S), medium (M), and long (L) terms. To cope with the scaling issue between the measured data and the CAMS' products, we developed a downscaling approach based on a long short-term memory (LSTM) neural network method which, apart from modeled ozone, also assimilated meteorological quantities as well as lagged O3 observations. Results show the benefit of applying the LSTM method instead of using the original CAMS products for providing O3 over Iran. It is found that lagged O3 observation has a larger contribution than other predictors in improving the LSTM. Compared with the S, the M component shows more associations with observations, e.g., correlation coefficients larger than 0.7 for the S and about 0.95 for the M in both models. The performance of the models varies across cities; for example, the highest error is for areas with high emissions of O3 precursors. The robustness of the results is confirmed by performing an additional downscaling method. This study demonstrates that coarse-scale global model data, such as CAMS, need to be downscaled for regulatory purposes or policy applications at local scales. Our method can be useful not only for the evaluation but also for the prediction of other chemical species, such as aerosols.

Enhancing GNSS Deformation Monitoring Forecasting with a Combined VMD-CNN-LSTM Deep Learning Model

Hydraulic infrastructures are susceptible to deformation over time, necessitating reliable monitoring and prediction methods. In this study, we address this challenge by proposing a novel approach based on the combination of Variational Mode Decomposition (VMD), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM) methods for Global Navigation Satellite Systems (GNSS) deformation monitoring and prediction modeling. The VMD method is utilized to decompose the complex deformation signals into intrinsic mode functions, which are then fed into a CNN method for feature extraction. The extracted features are input into an LSTM method to capture temporal dependencies and make predictions. The experimental results demonstrate that the proposed VMD-CNN-LSTM method exhibits an improvement by about 75%. This research contributes to the advancement of deformation monitoring technologies in water conservancy engineering, offering a promising solution for proactive maintenance and risk mitigation strategies.

Research on self-healing characteristic and state prediction method of the copper based powder metallurgy materials on friction interface

In high power density transmission systems, the friction and wear characteristic of copper based powder metallurgy materials is directly linked to working reliability. Moreover, these materials have frictional self-healing characteristic at the material interface. This paper focuses on exploring the healing mechanism of copper based powder metallurgy materials and conducts ‘damage-healing’ tests, proposing a method to characterize the self-healing characteristic. Subsequently, through comparative tests, the influence of temperature, speed, and pressure on the self-healing characteristics is analyzed. The results show that the increase in temperature reduces the furrow width and depth by 15.30% and 59.76%, respectively. Pressure has the greatest effect on surface roughness, reducing it by 67%. Meanwhile, this paper developed a PSO (Particle Swarm Optimization)-LSTM (Long Short-Term Memory) method to accurately predict the self-healing characterization parameters and self-healing time with small error (average 4.35%) and high correlation coefficient (R 2) (average 0.976). This study contributes to the development of interface repair technology for friction materials.

Classification of Intracranial Hemorrhage (CT) Images Using CNN- LSTM Method and Image-Based GLCM Features

In this study, a hybrid approach is utilized, which combines image-based grey-level co-occurrence matrix attributes and transformed-based attributes. The combination of attribute-based techniques is more accurate in categorizing cerebral hemorrhages in CT images when compared with the transformed attribute-based and image-attribute-based approaches. Deep learning methods, particularly long short-term memory (LSTM), have gained popularity in natural language processing applications like text and sentiment analysis. This paper presents an in-depth deep-learning system that is fully automated. The purpose of the system is to organize radiological data and identify cases of intracranial hemorrhage (ICH) for diagnosis. The structure of the proposed DL model includes 1D convolution neural networks (CNN), long short-term memory (LSTM) units, and a logistic function. A significant dataset consisting of 12,852 radiological reports of head computed tomography (CT) was used to train and assess these components.

Obstructive Sleep Apnea Syndrome Identification Using CNN-LSTM Hybrid Model

Obstructive Sleep Apnea (OSA) is a common sleep problem. It causes breathing issues during sleep. This disturbs oxygen intake and sleep patterns. Detecting OSA early and accurately is important for treatment. In our study, we look into using advanced deep learning for OSA detection. We focus on Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNN), and a new combined method. We also check the impact of different functions and methods on OSA detection accuracy. Our tests reveal some interesting facts. The Nadam optimizer gives the best OSA detection at 95.22% accuracy. The combined CNN and LSTM method achieves 95.38% accuracy. Using LSTM with the sigmoid function, we get a 93.80% accuracy. These results stress the importance of choosing suitable models and methods for OSA detection. Our study helps improve diagnostic tools for OSA. These findings can aid in early and improved treatment of OSA, giving people a better quality of life and health.

BO-STA-LSTM: Building energy prediction based on a Bayesian optimized spatial-temporal attention enhanced LSTM method

In predicting building energy (affected by seasons), there are issues like inefficient hyperparameter optimization and inaccurate predictions, it is unclear whether spatial and temporal attention improves performance. This study proposes a method based on Bayesian Optimization (BO), Spatial-Temporal Attention (STA), and Long Short-Term Memory (LSTM). Seven improved LSTM models (BO-LSTM, SA-LSTM, TA-LSTM, STA-LSTM, BO-SA-LSTM, BO-TA-LSTM, and BO-STA-LSTM) are compared with LSTM and the impacts of seasonal variations on BO-STA-LSTM are analysed using different sample types and time domain analysis. To further demonstrate the efficiency of the proposed method, comparisons with convolutional neural network (CNN) and temporal convolutional network (TCN) are performed, and followed by validation with new datasets. The findings indicate that adding STA and BO to LSTM enhances the average R2 of prediction performance by 0.0885. BO alone contributes 0.0717, while adding attention improves 0.0560. BO-STA-LSTM achieves higher prediction accuracy for similar test and training samples or a test sample size of 14,016, effectively capturing seasonal, trend, and peak energy patterns. Additionally, BO-STA-LSTM outperforms CNN and TCN, demonstrating superior prediction accuracy.

An Integrated Framework with ADD-LSTM and DeepLabCut for Dolphin Behavior Classification

Caring for dolphins is a delicate process that requires experienced caretakers to pay close attention to their behavioral characteristics. However, caretakers may sometimes lack experience or not be able to give their full attention, which can lead to misjudgment or oversight. To address these issues, a dolphin behavior analysis system has been designed to assist caretakers in making accurate assessments. This study utilized image preprocessing techniques to reduce sunlight reflection in the pool and enhance the outline of dolphins, making it easier to analyze their movements. The dolphins were divided into 11 key points using an open-source tool called DeepLabCut, which accurately helped mark various body parts for skeletal detection. The AquaAI Dolphin Decoder (ADD) was then used to analyze six dolphin behaviors. To improve behavior recognition accuracy, the long short-term memory (LSTM) neural network was introduced. The ADD and LSTM models were integrated to form the ADD-LSTM system. Several classification models, including unidirectional and bidirectional LSTM, GRU, and SVM, were compared. The results showed that the ADD module combined with a double-layer bidirectional LSTM method achieved high accuracy in dolphin behavior analysis. The accuracy rates for each behavior exceeded 90%.

Evaluation of LSTM and Wavelet Methods for Wind Speed Forecasting in Bahia, Brazil

The pursuit of sustainable development is intricately linked to the effective management of renewable energy resources, with wind energy emerging as a key player on the global stage. However, the inherent volatility and intermittency of wind patterns pose significant challenges to accurate wind speed forecasting, which is crucial for the stable operation of wind turbines. Our study presents a novel forecasting model that integrates cuttingedge data decomposition techniques with Long Short-Term Memory (LSTM) networks in this context. Our approach leverages advanced methodologies such as Wavelet Transform (WT), Empirical Mode Decomposition (EMD), and Enhanced Empirical Mode Decomposition (EEMD) to segment time series data into distinct high and low-frequency components. These segmented signals are then individually forecasted using Bidirectional LSTM (BiLSTM) networks, with the amalgamation of these predictions providing the final forecast output. Our empirical findings demonstrate that the hybrid model, particularly utilizing EMD and EEMD, exhibits superior performance compared to existing forecasting models in terms of both accuracy and stability. By effectively combining sophisticated data decomposition techniques with state-of-the-art deep learning algorithms, our proposed model offers a robust solution for wind speed forecasting. This facilitates the efficient management of renewable energy resources and advances the cause of sustainable development initiatives worldwide.

Sentiment Analysis on the Quality of Public Services with User Satisfaction Prediction of YuhSinau Application Managed by BKPSDM Kabupaten Kebumen Using LSTM Method

The quality of public services is critical in providing effective and responsive governance in an increasingly digital society. The development of the YuhSinau application by the Personnel and Human Resource Development Agency (BKPSDM) of Kebumen Regency offers an innovative response to the growing need for e-learning solutions for local government civil servants (Pegawai Negeri Sipil or PNS). However, determining the app’s effectiveness and user satisfaction is critical. This demands a thorough sentiment analysis in order to acquire insights into users’ thoughts and opinions about the quality of public services supplied by YuhSinau. The Long Short-Term Memory (LSTM) approach is used in this article to examine feelings and forecast customer pleasure. Data collection from multiple sources, initial data preprocessing, LSTM model construction, training, validation, and prediction are all part of the process. The results show that the model has some drawbacks, most notably its failure to appropriately explain variation in target data due to a negative R-squared value. Enhancements to the LSTM architecture, hyperparameter adjustment, and the use of more diverse and representative training data are proposed to improve the model. Continuous review and responsiveness to user comments are critical for improving the quality of public services via the YuhSinau application.
 Keywords: sentiment analysis, public service quality, user satisfaction, YuhSinau application,LSTM method

Comparative Analysis of Commonly Used Machine Learning Approaches for Li-Ion Battery Performance Prediction and Management in Electric Vehicles

The significant role of Li-ion batteries (LIBs) in electric vehicles (EVs) emphasizes their advantages in terms of energy density, being lightweight, and being environmentally sustainable. Despite their obstacles, such as costs, safety concerns, and recycling challenges, LIBs are crucial in terms of the popularity of EVs. The accurate prediction and management of LIBs in EVs are essential, and machine learning-based methods have been explored in order to estimate parameters such as the state of charge (SoC), the state of health (SoH), and the state of power (SoP). Various machine learning techniques, including support vector machines, decision trees, and deep learning, have been employed for predicting LIB states. This study proposes a methodology for comparative analysis, focusing on classical and deep learning approaches, and discusses enhancements to the LSTM (long short-term memory) and Bi-LSTM (bidirectional long short-term memory) methods. Evaluation metrics such as MSE, MAE, RMSE, and R-squared are applied to assess the proposed methods’ performances. The study aims to contribute to technological advancements in the electric vehicle industry by predicting the performance of LIBs. The structure of the rest of the study is outlined, covering materials and methods, LIB data preparation, analysis, the proposal of machine learning models, evaluations, and concluding remarks, with recommendations for future studies.

State of charge estimation for Li-ion battery during dynamic driving process based on dual-channel deep learning methods and conditional judgement

The accurate estimation of the state of charge (SOC) is crucial for the safe and reliable operation of Li-ion batteries. In this paper, novel dual-channel deep learning methods were proposed for SOC estimation during dynamic driving cycles. For dual-channel deep learning methods, each channel of the deep learning methods could be a single convolutional neural network (CNN) layer, single long short-term memory (LSTM) layer or sequentially combined structure of CNN and LSTM. After evaluation, it could be found that dual-channel deep learning methods could achieve higher accuracy than conventional single-channel method, however, the model size and floating-point operations (FLOPs) could be either reduced or increased. Among all the dual-channel deep learning methods, CNN + LSTM-CNN + LSTM method could achieve the highest accuracy and the shortest training time with sacrifices in computational complexity. CNN–CNN + LSTM method could achieve higher accuracy than the single-channel deep learning method with reduced computational complexity and model size. As the error of SOC estimation is highly correlated with voltage signals, conditional judgement was then integrated with dual-channel deep learning method to obtain higher accuracy. After integrated with conditional judgement, the averaged error of SOC estimation could be reduced by more than 45% without a substantial sacrifice in computational time.

A Combined Method for Short-Term Load Forecasting Considering the Characteristics of Components of Seasonal and Trend Decomposition Using Local Regression

In response to the complexity and high volatility of original load data affecting the accuracy of load forecasting, a combined method for short-term load forecasting considering the characteristics of components of seasonal and trend decomposition using local regression (STL) is proposed. The original load data are decomposed into a trend component, seasonal component, and residual component using STL. Then, considering the characteristics of each component, a long short-term memory (LSTM) neural network, a convolutional neural network (CNN), and Gaussian process regression (GPR) are used to predict the trend component, seasonal component, and residual component, respectively. The final outcome of the load forecasting is obtained by summing the forecasted results of each individual component. A specific case study is conducted to compare the proposed combined method with LSTM, CNN, GPR, STL-LSTM, STL-CNN, and STL-GPR prediction methods. Through comparison, the proposed combined method exhibits lower errors and higher accuracy, demonstrating the effectiveness of this method.

Prediction model of land surface settlement deformation based on improved LSTM method: CEEMDAN-ICA-AM-LSTM (CIAL) prediction model.

The uneven settlement of the surrounding ground surface caused by subway construction is not only complicated but also liable to cause casualties and property damage, so a timely understanding of the ground settlement deformation in the subway excavation and its prediction in real time is of practical significance. Due to the complex nonlinear relationship between subway settlement deformation and numerous influencing factors, as well as the existence of a time lag effect and the influence of various factors in the process, the prediction performance and accuracy of traditional prediction methods can no longer meet industry demands. Therefore, this paper proposes a surface settlement deformation prediction model by combining noise reduction and attention mechanism (AM) with the long short-term memory (LSTM). The complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and independent component analysis (ICA) methods are used to denoise the input original data and then combined with AM and LSTM for prediction to obtain the CEEMDAN-ICA-AM-LSTM (CIAL) prediction model. Taking the settlement monitoring data of the construction site of Urumqi Rail Transit Line 1 as an example for analysis reveals that the model in this paper has better effectiveness and applicability in the prediction of surface settlement deformation than multiple prediction models. The RMSE, MAE, and MAPE values of the CIAL model are 0.041, 0.033 and 0.384%; R2 is the largest; the prediction effect is the best; the prediction accuracy is the highest; and its reliability is good. The new method is effective for monitoring the safety of surface settlement deformation.

SOC Estimation of Li-Po Battery Using Machine Learning and Deep Learning Methods

The aviation industry is one of the most important areas where developing technology contributes. It is important to evaluate many factors for the safe and comfortable flight of unmanned aerial vehicles (UAVs), one of the most popular areas of this industry. One of the most important of these factors is flight time estimation. Battery state of charge (SOC) plays a big role in flight time estimation. In this study, using the data obtained from the tests carried out using a lithium-polymer battery in the electric UAV engine test equipment, the SOC of the battery was estimated using deep learning like as Long-Short Term Memory (LSTM) and machine learning methods like as Support Vector Regression (SVR) and Random Forest (RF). The main reason why these methods are preferred is that they are suitable for time series analysis in the forecasting process, are trained faster, and have generalization abilities. The proposed models were compared among themselves and the simulation results were presented with graphs and tables.
 When the results are examined, the predicted values and true values are quite compatible. This shows that the proposed methods can be used effectively in SOC estimation.

A new denoising approach based on mode decomposition applied to the stock market time series: 2LE-CEEMDAN.

Time series, including noise, non-linearity, and non-stationary properties, are frequently used in prediction problems. Due to these inherent characteristics of time series data, forecasting based on this data type is a highly challenging problem. In many studies within the literature, high-frequency components are commonly excluded from time series data. However, these high-frequency components can contain valuable information, and their removal may adversely impact the prediction performance of models. In this study, a novel method called Two-Level Entropy Ratio-Based Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (2LE-CEEMDAN) is proposed for the first time to effectively denoise time series data. Financial time series with high noise levels are utilized to validate the effectiveness of the proposed method. The 2LE-CEEMDAN-LSTM-SVR model is introduced to predict the next day's closing value of stock market indices within the scope of financial time series. This model comprises two main components: denoising and forecasting. In the denoising section, the proposed 2LE-CEEMDAN method eliminates noise in financial time series, resulting in denoised intrinsic mode functions (IMFs). In the forecasting part, the next-day value of the indices is estimated by training on the denoised IMFs obtained. Two different artificial intelligence methods, Long Short-Term Memory (LSTM) and Support Vector Regression (SVR), are utilized during the training process. The IMF, characterized by more linear characteristics than the denoised IMFs, is trained using the SVR, while the others are trained using the LSTM method. The final prediction result of the 2LE-CEEMDAN-LSTM-SVR model is obtained by integrating the prediction results of each IMF. Experimental results demonstrate that the proposed 2LE-CEEMDAN denoising method positively influences the model's prediction performance, and the 2LE-CEEMDAN-LSTM-SVR model outperforms other prediction models in the existing literature.

A Data-Driven Prediction Method for Proton Exchange Membrane Fuel Cell Degradation

This paper proposes a long short-term memory (LSTM) network to predict the power degradation of proton exchange membrane fuel cells (PEMFCs), and in order to promote the performance of the LSTM network, the ant colony algorithm (ACO) is introduced to optimize the hyperparameters of the LSTM network. First, the degradation mechanism of PEMFCs is analyzed. Second, the ACO algorithm is used to set the learning rate and dropout probability of the LSTM network combined with partial aging data, which can show the characteristics of the dataset. After that, the aging prediction model is built by using the LSTM and ACO (ACO-LSTM) method. Moreover, the convergence of the method is verified with previous studies. Finally, the fuel cell aging data provided by the Xiangyang Da’an Automotive Testing Center are used for verification. The results show that, compared with the traditional LSTM network, ACO-LSTM can predict the aging process of PEMFCs more accurately, and its prediction accuracy is improved by about 35%, especially when the training data are less. At the same time, the performance of the model trained by ACO-LSTM is also excellent under other operating conditions of the same fuel cell, and it has strong versatility.

Short-Term Prediction of the Intermediate Point Temperature of a Supercritical Unit Based on the EEMD–LSTM Method

The quality of the intermediate point temperature control of a supercritical unit is directly related to the quality of the coal–water ratio and main steam temperature control of the supercritical unit, which is also related to the economy and safety of the unit. In order to improve the accuracy of short-term predictions of the intermediate point temperature, a short-term prediction model of the intermediate point temperature based on the EEMD (Ensemble Empirical Mode Decomposition)-LSTM (Long Short-Term Memory) model is proposed. This model uses the data of a 600 MW thermal power station in 2022 as a sample. The EEMD method is used to decompose the historical data into IMF components and residual components, and the correlation between each component and the original data is calculated. The relevant components are sent to the LSTM neural network, and all the sub-components are superimposed to obtain the final intermediate point temperature prediction results. At the same time, the BP and LSTM models are built to compare the errors with the proposed model. The results show that the single model will produce large errors when predicting the factors of large data fluctuations. The EEMD–LSTM coupling model can fully extract the detailed features and the prediction effect is obvious. The prediction accuracy of the EEMD–LSTM prediction model built in this paper is significantly better than that of the other two models. It has certain application value in the research field of intermediate point temperature prediction and can meet the requirements of short-term predictions of the intermediate point temperature.

인공지능 기반 심리상담에 활용되는 딥러닝 기법 고찰

Objectives This study aims to review and summarize deep learning techniques that have been developed to date and to discuss how they can be applied in practice to artificial intelligence-based psychology counseling robots.
 Methods To examine the latest deep learning techniques utilized in artificial intelligence psychological counseling, an extensive literature search was conducted for research papers and relevant books from both domestic and in-ternational sources in the fields of emotion analysis, facial expression generation, and text generation.. For do-mestic literature, databases such as RISS and KISS were utilized, and for international literature, research paper search sites including ERIC (ProQuest), JSTOR, and Google Scholar were employed.
 Results When applying artificial intelligence in psychological counseling, the prominent technology of interest in-volves recognizing emotions from the interlocutor's facial expressions and generating similar expressions, or rec-ognizing emotions from sentences spoken by the interlocutor and generating responsive sentences. The study in-troduced the concepts of machine learning and deep learning and explained the workings of deep learning techni-ques, such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short Term Memory), Transformer, BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pretrained Transformer), and GAN (Generative Adversarial Network). These techniques are founded on the princi-ples derived from human brain functioning. Particularly, emotion recognition programs through image analysis have predominantly relied on CNN, while research in the realm of language has advanced with the utilization of GPT-2, LSTM, and similar methods.
 Conclusions Among the existing deep learning techniques, the recognition of emotions from the interlocutor's fa-cial expressions and the generation of similar expressions, or the recognition of emotions from sentences spoken by the interlocutor and the generation of responsive sentences, are the technologies predominantly under scrutiny for their application in artificial intelligence-based psychological counseling. Nevertheless, rule-based chatbots continue to dominate the landscape. This paper explored the merits and demerits of AI-based counseling em-ploying such deep learning techniques. Looking ahead, there is a need for the advancement of technology that en-ables natural interactions, coupled with the essential verification of its effectiveness.

Predicting Steam Turbine Power Generation: A Comparison of Long Short-Term Memory and Willans Line Model

This study focuses on using machine learning techniques to accurately predict the generated power in a two-stage back-pressure steam turbine used in the paper production industry. In order to accurately predict power production by a steam turbine, it is crucial to consider the time dependence of the input data. For this purpose, the long-short-term memory (LSTM) approach is employed. Correlation analysis is performed to select parameters with a correlation coefficient greater than 0.8. Initially, nine inputs are considered, and the study showcases the superior performance of the LSTM method, with an accuracy rate of 0.47. Further refinement is conducted by reducing the inputs to four based on correlation analysis, resulting in an improved accuracy rate of 0.39. The comparison between the LSTM method and the Willans line model evaluates the efficacy of the former in predicting production power. The root mean square error (RMSE) evaluation parameter is used to assess the accuracy of the prediction algorithm used for the generator’s production power. By highlighting the importance of selecting appropriate machine learning techniques, high-quality input data, and utilising correlation analysis for input refinement, this work demonstrates a valuable approach to accurately estimating and predicting power production in the energy industry.