Long Short-term Memory Neural Network Research Articles

A New Epileptic Seizure Prediction Framework Based on Electroencephalography Signals

This research seeks to evaluate how effectively seizures can be predicted and managed in epilepsy using a specialized deep learning model based on Long Short-Term Memory (LSTM) neural networks. The model leverages non-invasive scalp electroencephalography (EEG) recordings for predicting seizures. To develop and assess the proposed LSTM neural network model, a comprehensive dataset was gathered. The model emphasizes achieving high sensitivity and reducing false alarms to improve its real-time applicability. The evaluation involved various metrics to measure accuracy, sensitivity, and rates of false positives and false negatives. The effectiveness of the proposed LSTM neural network model was outstanding, with accuracy rates ranging from 99.07% to 99.95%. Notably, the sensitivity score of 1 confirmed precise prediction for all seizure cases. The model demonstrated minimal false positive and false negative rates, highlighting its reliability in predicting seizures. This study emphasizes the promising potential of the proposed LSTM neural network model in providing advanced warning for seizures. The high accuracy and sensitivity rates suggest its usefulness in enabling timely preventive measures for patients, ultimately reducing the occurrence of seizures. This innovative approach holds significance in enhancing the overall management and quality of life for individuals dealing with epilepsy.

Soft sensor modeling method and application based on TSECIT2FNN-LSTM

To address the issue of low accuracy in soft sensor modeling of key variables caused by multi-variable coupling and parameter sensitivity in complex processes, this paper introduces a TSK-type-based self-evolving compensatory interval type-2 fuzzy Long short-term memory (LSTM) neural network (TSECIT2FNN-LSTM) soft sensor model. The proposed TSECIT2FNN-LSTM integrates the LSTM neural network with the interval type-2 fuzzy inference system to address long-term dependencies in sequence data by utilizing the gate mechanism of the LSTM neural network. The TSECIT2FNN-LSTM structure learning algorithm uses the firing strength of the network rule antecedent to decide whether to generate new rules to improve the rationality of the network structure. TSECIT2FNN-LSTM parameter learning utilizes the gradient descent method to optimize network parameters. However, unlike other interval type-2 fuzzy neural network gradient calculation processes, the error term in the LSTM node parameter gradient of TSECIT2FNN-LSTM is propagated backwards in the time dimension. Additionally, the error term is simultaneously transferred to the upper layer network to enhance network prediction accuracy and memory capabilities. The TSECIT2FNN-LSTM soft sensor model is utilized to predict the alcohol concentration in wine and the nitrogen oxide emission in gas turbines. Experimental results demonstrate that the proposed TSECIT2FNN-LSTM soft sensing model achieves higher prediction accuracy compared to other models.

Research on risk detection and management based on machine learning

With the rapid advancement in financial risk management, machine learning techniques are playing an increasingly crucial role in credit assessment and risk detection. They provide financial institutions with more scientific and precise platforms and methodologies for risk management. This paper explores various traditional machine learning methods in risk assessment, introduces the XGBoost ensemble learning method, and integrates traditional machine learning with Long Short-Term Memory (LSTM) neural networks to enhance the generalization capability of risk control and credit assessment models. This approach offers new perspectives and improvement directions for risk assessment standards and practices in the financial industry, affirming the potential application of machine learning technologies in future risk management.The results show that the random forest and decision tree have excellent accuracy and recall rates for distinguishing fraudulent transactions. After the introduction of LSTM neural networks, the accuracy and recall rates of fraudulent transaction recognition models reach around 99%, indicating that these models have good adaptability for fraud risk recognition.

Empowering Digital Civility with an NLP Approach for Detecting Twitter Cyberbullying through Boosted Ensembles

As the number of social networking sites grows, so do cyber dangers. Cyberbullying is harmful behaviour that uses technology to intimidate, harass or harm someone, often on social media platforms like Twitter. Machine learning is the optimal approach for cyberbullying detection on Twitter to process large amounts of data, identify patterns of offensive behaviour, and automate the detection process for corpus of tweets. To identify cyber threats using a trained model, the Boosted Ensemble (BE) technique is assessed with various machine learning algorithms such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Naive Bayes (NB), Decision Tree (DT), Support Vector Machine (SVM), Bidirectional LSTM (BILSTM), Recurrent Neural Network LSTM (RNN-LSTM), Multi-Modal Cyberbullying Detection (MMCD) and Random Forest (RF). These classifiers are trained on the vectorized data to classify the tweets to identify cyberbullying threats. The proposed framework can detect cyberbullying cases precisely on tweets. The significance of the work lies in detecting and mitigating cyber threats in real-time and it impacts in enhancing the safety and well-being of social media users by reducing instances of cyberbullying and other cyber threat. The comparative analysis is done using metrices like accuracy, precision, recall and F1-score and the comparison results shows that boosted ensemble technique outperforms other compared algorithms with its overall performance. The accuracy rate of CNN, LSTM, NB, DT, SVM, RF, BILSTM and BE (92.5%, 93.5%, 84.6%, 88%, 89.3%, 92%, 93.75% and 96%), precision rate of CNN, LSTM, NB, DT, SVM, RF, RNN-LSTM and BE (90.2%, 91.3%, 88%, 85%, 86%, 91.6%, 92.1% and 94%), recall rate of CNN, LSTM, NB, DT, SVM, RF, BILSTM and BE (89.8%, 90.7%, 90%, 82%, 88.67%, 89%, 91.04% and 93.7%) and F1 score of CNN, LSTM, NB, DT, SVM, RF, MMCD and BE (90.6%, 91.8%, 85%, 84.56% 87.2%, 90%, 84.6% and 94.89%).

Condition assessment and predictive maintenance for contact probe using health index and encoder‐decoder LSTM model

AbstractContact probe is broadly used for the continuous monitoring of microelectronic components in manufacturing industries. False rejection of fine product due to defective contact probe significantly reduces the yield in production. Traditionally, defect detection for contact probes heavily depends on a valid range manually defined by engineers over the measured value of certain parameters. However, the subjective range defined according to engineer experience is prone to trigger a high rate of false alarms due to the inherent noise in the measured parameters. To address this issue, we construct a health index (HI) with the contact resistance‐directly‐related features to help monitor and assess the condition of contact probe. Based on the established HI, we develop Long Short‐Term Memory (LSTM) encoder‐decoder machine learning model to assess the condition of contact probe by forecasting the HI value in the future. Encoders from LSTM and convolutional neural network (CNN) are selected as the encoder‐decoder architecture for the sequence‐to‐sequence prediction due to their advantage in extracting the correlation of features at different scales. An explainable Artificial Intelligence (XAI) technique named Local interpretable model‐agnostic explanations (LIME) is used to quantify the contribution of each feature to the model prediction. The encoder from CNN is found to outperform the LSTM encoder in extracting the inter‐feature correlation. Finally, the predicted HI is used to signal the alarm for the maintenance action of contact probe when its value is below a predefined threshold. Comparison between the action alarm triggered by the developed HI and the actual maintenance records suggests that the proposed approach achieves at least 75% accuracy for the triggered alarm in the next 15 mins.

Evaporation temperature prediction of the refrigerant-direct convective-radiant cooling system based on LSTM neural network

The existing radiant cooling systems are effective at handling indoor sensible heat load. However, the indoor latent load needs to be treated with an additional dehumidification system, which will increase the complexity and initial investment of the system. To settle this dilemma, a novel aluminum column-wing type refrigerant-direct convective-radiant cooling (ACT-RCC) system was proposed, which adopted refrigerant as direct medium and provided cooling and independent dehumidification simultaneously. Experiments were conducted in an enthalpy difference laboratory with the indoor air temperature of 22.0 °C − 30.0 °C and relative humidity of 60.0%. A numerical model of the ACT-RCC terminal was established and validated with the experimental data. To meet the indoor heat and latent load of residential buildings, a heuristic approach based on Long Short-Term Memory (LSTM) neural network for predicting the evaporation temperature of this system was proposed. Results showed that the predicted values were accurate enough by measured data and the maximum deviation of the evaporation temperature was 0.7 °C. The relationship between the evaporation temperature, indoor air temperature and relative humidity, and heat moisture ratio was derived. Besides, the indoor thermal comfort of the selected bedroom was also evaluated. Results showed that when the indoor air temperature and relative humidity remained 26.0 °C and 40.0% − 70.0%, the thermal comfort of this bedroom can achieve Level Ⅰ by adjusting the evaporation temperature of the ACT-RCC system of 2.0 °C − 19.8 °C.

Adaptive Neural Control for Hysteretic Nonlinear Systems With Hysteresis Neural Direct Inverse Compensator and Its Application.

Aiming at high precision control for a class of hysteretic nonlinear systems, a new hysteresis direct inverse compensator-based adaptive output feedback control scheme is designed in this article. First, a novel long short-term memory neural network (LSTMNN)-based hysteresis inverse compensator is established to compensate the asymmetric hysteresis nonlinearity, where the LSTMNN is used as the prediction mechanism for model operator weights, rather than the overall mapping of hysteresis input and output. Second, by designing the modified high-gain K-Filter states observer and the error transformed function, the unmeasurable states are estimated with arbitrarily small estimation error and the prespecified tracking performance is achieved. Lastly, the biconical dielectric elastomer actuator (DEA) motion platform is constructed. Then, the effectiveness of the proposed LSTMNN-based hysteresis inverse compensator and control scheme are verified on the experimental platform. The experimental results illustrate the effectiveness and advantages of proposed control scheme.

Mpox outbreak: Time series analysis with multifractal and deep learning network.

This article presents an overview of an mpox epidemiological situation in the most affected regions-Africa, Americas, and Europe-tailoring fractal interpolation for pre-processing the mpox cases. This keen analysis has highlighted the irregular and fractal patterns in the trend of mpox transmission. During the current scenario of public health emergency of international concern due to an mpox outbreak, an additional significance of this article is the interpretation of mpox spread in light of multifractality. The self-similar measure, namely, the multifractal measure, is utilized to explore the heterogeneity in the mpox cases. Moreover, a bidirectional long-short term memory neural network has been employed to forecast the future mpox spread to alert the outbreak as it seems to be a silent symptom for global epidemic.

Compensation Algorithm for Wind Turbine Main Shaft Loads Based on LSTM

Abstract For wind turbine development, load size is the core and critical factor. Currently, most companies employ Bladed software for load calculations, which enjoys widespread use in engineering applications due to its efficiency and ease of use. However, constrained by the limitations of blade element theory and momentum theory, there still exist discrepancies between the simulation results and actual conditions. This paper aims to combine simulation algorithms with the application of an LSTM (Long Short-Term Memory) neural network to develop a compensation algorithm for the main shaft loads of wind turbines. By doing so, it endeavors to realize real-time monitoring of the main shaft loads, thereby providing a foundation for achieving high-precision control in wind turbine systems.

Dynamic Prediction for Pollutant Emissions of Coal-fired Power Plant Based on CNN-LSTM-Attention

Abstract The prediction accuracy of pollutant emissions by using traditional modeling methods is unsatisfactory in dynamic conditions. To overcome the problem, data-driven modeling was introduced to build the dynamic model of pollutant emissions of power plants in this paper. Combining with the running data of a 300MW circulating fluidized bed (CFB) unit, the dynamic prediction models of SO2 and NO x emissions were established respectively by using conventional neural network-long short-term memory and attention mechanism (CNN-LSTM-Attention). Moreover, LSSVM, LSTM and CNN-LSTM were introduced for comparison to demonstrate the superiority of CNN-LSTM-Attention model respectively. Simulation results indicate that model can imitate change trend of actual data with high accuracy over a long period of time. Compared with LSSVM, LSTM and CNN-LSTM, the proposed model has better modeling performance under different load conditions. This work provides certain guidance for the application of deep learning in the industrial field.

Enhanced Foreign Exchange Volatility Forecasting Using Ceemdan with Optuna-Optimized Ensemble Deep Learning Model

Foreign Exchange (FX) is the largest financial market in the world, with a daily trading volume that significantly exceeds that of stock and futures markets. The prediction of FX volatility is a critical financial concern that has garnered significant attention from researchers and practitioners due to its far-reaching implications in the financial markets. This paper presents a novel hybrid ensemble forecasting model integrating a decomposition strategy and three deep learning (DL) models: Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and Convolutional Neural Network (CNN). This combination addresses individual models' limitations and further improves the accuracy and stability of FX volatility forecasting. The proposed approach utilizes the CEEMDAN technique to decompose volatility into multiple distinct intrinsic mode functions (IMFs) and merges these IMFs with GARCH and EGARCH volatilities to form the input dataset for the DL models. In addition, we employed an attention mechanism to improve the effectiveness of the DL techniques. Furthermore, the hyperparameters for the DL models are optimized using the Optuna algorithm. Finally, a hybrid ensemble model for forecasting exchange rate volatility is developed by combining the predictions of three distinct DL models. The proposed approach is evaluated against various benchmark models using evaluation measures such as MSE, MAE, HMSE, HMAE, RMSE, Q-LIKE, and the model confidence set (MCS) approach. The results demonstrate that our proposed approach provides accurate and reliable forecasts of FX volatility under different forecasting regimes, making it a valuable tool for financial practitioners and researchers.

Analysis of rogue wave in the mid-infrared supercontinuum under femtosecond weak seed pulse conditions based on deep learning

The generation process of rogue wave (RW) is affected by noise, which is an unstable state, and the existence of RW will reduce the stability of mid-infrared supercontinuum. However, the process of studying RW requires a large amount of data simulation and statistics, and traditional methods are time-consuming and inefficient. Therefore, this paper adopts long short-term memory (LSTM) neural network to obtain the spectrum information after transmission for a certain distance according to the waveform information of the incident pulse. The results show that the LSTM neural network structure can train and predict the peak power, time deviation information, time intensity evolution and spectrum evolution of RW after 10 cm propagation with only changing the number of internal units. And it performs well on both large and small data sets.

Displacement Interval Prediction Method for Arch Dam with Cracks: Integrated STL, MF-DFA and Bootstrap

Displacement prediction models based on measured data have been widely applied in structural health monitoring. However, most models neglect the particularity of displacement monitoring for arch dams with cracks, nor do they thoroughly analyze the non-stationarity and uncertainty of displacement. To address this issue, the influencing factors of displacement were first considered, with crack opening displacement being incorporated into them, leading to the construction of the HSCT model that accounts for the effects of cracks. Feature selection was performed on the factors of the HSCT model utilizing the max-relevance and min-redundancy (mRMR) algorithm, resulting in the screened subset of displacement influence factors. Next, displacement was decomposed into trend, seasonal, and remainder components applying the seasonal-trend decomposition using loess (STL) algorithm. The multifractal characteristics of these displacement components were then analyzed by multifractal detrended fluctuation analysis (MF-DFA). Subsequently, displacement components were predicted employing the convolutional neural network-long short-term memory (CNN-LSTM) model. Finally, the impact of uncertainty factors was quantified using prediction intervals based on the bootstrap method. The results indicate that the proposed methods and models are effective, yielding satisfactory prediction accuracy and providing scientific basis and technical support for the health diagnosis of hydraulic structures.

Pre-compensating digital-to-analog converter impairments with an LSTM neural network

The performance of high-speed coherent communication heavily depends on digital-to-analog converters (DACs) when they operate within bandlimited and nonlinear regimes. We examine neural network (NN) based digital pre-distortion (DPD) methods to compensate the memory effects that are intertwined with the nonlinear response of the DAC. Using a long short-term memory (LSTM) NN architecture, we examine 8-level amplitude shift keying (ASK-8) at 64 Gbaud. We demonstrate experimental improvements of 1.6 dB and 2 dB in signal-to-noise ratio (SNR) compared to the Volterra and linear solutions, respectively.

Optimal Sensor Placement and Multimodal Fusion for Human Activity Recognition in Agricultural Tasks

This study examines the impact of sensor placement and multimodal sensor fusion on the performance of a Long Short-Term Memory (LSTM)-based model for human activity classification taking place in an agricultural harvesting scenario involving human-robot collaboration. Data were collected from twenty participants performing six distinct activities using five wearable inertial measurement units placed at various anatomical locations. The signals collected from the sensors were first processed to eliminate noise and then input into an LSTM neural network for recognizing features in sequential time-dependent data. Results indicated that the chest-mounted sensor provided the highest F1-score of 0.939, representing superior performance over other placements and combinations of them. Moreover, the magnetometer surpassed the accelerometer and gyroscope, highlighting its superior ability to capture crucial orientation and motion data related to the investigated activities. However, multimodal fusion of accelerometer, gyroscope, and magnetometer data showed the benefit of integrating data from different sensor types to improve classification accuracy. The study emphasizes the effectiveness of strategic sensor placement and fusion in optimizing human activity recognition, thus minimizing data requirements and computational expenses, and resulting in a cost-optimal system configuration. Overall, this research contributes to the development of more intelligent, safe, cost-effective adaptive synergistic systems that can be integrated into a variety of applications.

Research on Depression Recognition Based on University Students’ Facial Expressions and Actions with the Assistance of Artificial Intelligence

As artificial intelligence (AI) technology advances, its application in the field of psychology has witnessed significant advancements. In this paper, with the assistance of AI, 80 university students with depression and 80 university students with normal psychology were selected as the subjects. The facial expression feature data were extracted through OpenFace, and the action feature data were extracted based on a Kinect camera. Then, the convolutional neural network-long short-term memory (CNN-LSTM) and temporal convolutional neural network (TCN) approaches were designed for recognition. Finally, a weighted fusion recognition method was proposed. The results showed that compared with the support vector machine, back-propagation neural network, and other approaches, the CNN-LSTM and TCN methods showed better performance in the recognition of single feature data, and the accuracy reached 0.781 and 0.769, respectively. After weighted fusion, the accuracy reached the highest at 0.875. The results verify that the methods designed in this paper are effective in identifying depressive emotions through facial expressions and actions among university students and have the potential for practical application.

Automatic detection and interpretable analysis of learners’ cognitive states based on electroencephalogram signals

The development of higher-order thinking skills (HOTS) in learners is one of the educational objectives of the 21st century. Detecting learners’ higher and lower cognitive states based on electroencephalogram (EEG) signals is crucial for promoting the development of HOTS. In this study, we investigated the feasibility of using an EEG-based deep learning (DL) model to classify higher and lower cognitive states and employed eXplainable Artificial Intelligence (XAI) techniques to examine the neural mechanisms associated with the cognitive states. To trigger learners’ cognitive states, both high-level and low-level learning activities were developed based on a revised Bloom's taxonomy and the Interactive-Constructive-Active-Passive (ICAP) framework and used with 22 subjects whose EEG signal was recorded and detected with the Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) model. The CNN-LSTM model yielded a remarkable recognition accuracy of around 99 %. Finally, we proposed LIME-Brain Area (LIME-BA), an improvement of Local Interpretable Model-agnostic Explanation (LIME), to identify the distinctive attributes of brain area activities for different levels of cognitive states. According to the XAI interpretable analysis, we found that the frontal and temporal areas were activated in a higher cognitive state, and the occipital and parietal regions were activated in a lower cognitive state. This study provides further evidence for educators to design cognitive-guided instructional activities to enhance learners’ development of HOTS.

Nonlinear Ensemble Deep Learning Model for Energy Consumption Prediction with Bayesian Optimization

Accurate prediction of electric energy consumption is crucial for efficient load dispatching, energy utilization, and grid operation. Traditional statistical and classical machine learning methods struggle with the nonlinear nature of energy consumption data, often leading to higher prediction errors. Additionally, deep learning models using a single approach face challenges such as convergence to local minima and poor generalization. This paper proposes a nonlinear ensemble deep learning model for residential energy consumption prediction, incorporating Bayesian optimization for hyperparameter tuning. The model combines Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and 1D Convolutional Neural Networks (1D-CNN), leveraging their powerful nonlinear feature learning capabilities. A k-means clustering approach is used to preprocess and reduce variability in the data, enhancing the ensemble model's performance. The ensemble model was tested on real energy consumption data from two districts in Addis Ababa, showing significant improvements in prediction accuracy with lower MAE, RMSE, and MAPE values compared to single models and un-clustered data. The integration of clustering and Bayesian optimization further enhanced model generalizability and minimized overfitting, demonstrating the effectiveness of a nonlinear approach in capturing complex energy consumption patterns.

Intelligent aeration amount prediction control for wastewater treatment process based on recurrent neural network

The use of machine learning in artificial intelligence to solve industrial problems has been a current trend. Predictions can be made by machine learning algorithms. This greatly improves the efficiency and also the accuracy. With the development of industrialization, the pollution of water resources is becoming more and more serious. How to treat wastewater more cost-effectively to meet the discharge standards has become one of the most urgent problems in the world. In the Anaerobic-Anoxic-Aerobic (AAO) process, the key to reach the standard of sewage treatment with low cost and high efficiency lies in the aeration link. In this study, multiple machine learning methods are used for modeling, and proposed a method employing the long short-term memory (LSTM) neural network model for regression prediction addresses the need for a more accurate prediction approach. By utilizing data from the preceding 100 days, this model replaces manual, experience-based prediction methods, thereby mitigating energy consumption. This model is optimized by training and testing with real wastewater treatment plant data and continuously adjusting the parameters through multiple sets of experiments. This can make the real recorded aeration amount and the actual aeration amount infinitely close. Through the experimental comparison, it is found that the performance of the LSTM neural network model is better, with about 14% higher accuracy than the benchmark model.

Energy storage battery state of health estimation based on singular value decomposition for noise reduction and improved LSTM neural network

Accurate estimation of the State of Health (SOH) of batteries is important for intelligent battery management in energy storage systems. To solve the problems of poor quality of data features as well as the difficulty of model parameter adjustment, this study proposes a method for estimating the SOH of lithium batteries based on denoising battery health features and an improved Long Short-Term Memory (LSTM) neural network. First, in this study, three health features related to SOH decrease were selected from the battery charge/discharge data, and the singular value decomposition technique was applied to the noise reduction of the features to improve their correlation with the SOH. Then, the whale optimization algorithm is improved using cubic chaotic mapping to enhance its global optimization-seeking capability. Then, the Improved Whale Optimization Algorithm (IWOA) is used to optimize the model parameters of LSTM, and the IWOA-LSTM model is applied to the battery SOH estimation. Finally, the model proposed in this research is validated against the Center for Advanced Life Cycle Engineering (CALCE) battery dataset. The experimental results show that the prediction error of battery SOH by the method proposed in this study is less than 0.96%, and the prediction error is reduced by 49.42% compared to its baseline model. The method presented in the article achieves accurate estimation of the SOH, providing a reference for practical engineering applications.