Long Short-term Memory Neural Network Research Articles

Core temperature estimation of lithium-ion battery based on numerical model fusion deep learning

Temperature has a critical impact on the lifespan and safety of lithium batteries. This paper proposes a battery core temperature estimation method based on numerical model fused with long short-term memory (LSTM) neural network. The proposed technique extracts features from the numerical model, estimates the volume-averaged temperature by electrochemical impedance spectroscopy (EIS), uses an LSTM neural network to learn thermodynamic parameters and complex calculations, which takes advantage of the strengths of each method, and achieves accurate core temperature estimation. The effects of state of charge (SOC) and temperature on EIS are explored, impedance properties are selected on the criteria of robustness and rapidity, and the estimation of the volume-averaged temperature is achieved using the imaginary part of the impedance. The proposed method can achieve root mean squared error (RMSE) of less than 0.28 °C and mean absolute error (MAE) of less than 0.23 °C. The proposed method has advantages of high estimation accuracy and does not require an electrothermal model. It also considers the effect of ambient temperature and has a good generalization capability.

A Predictive Approach for Lithium-Ion Battery SOH using LSTM Neural Networks Enhanced by Health Matrix Optimization

The State of Health (SOH) is a critical performance metric that characterizes the condition of lithium-ion batteries, directly influencing their lifespan and operational efficiency. In order to enhance the accuracy of SOH predictions for batteries and reduce operational risks, a novel approach has been introduced. This method, based on Health Matrix Optimization for Long Short Term Memory (LSTM) neural networks, aims to optimize the prediction process. Initially, the Spearman correlation coefficient method is employed to analyze the correlation of battery state data. Through the use of a heatmap, data points with strong correlations to SOH are identified, leading to the creation of a health feature matrix. This matrix is then utilized to fine-tune the hyperparameters of the LSTM neural network, resulting in refined approximations. Subsequently, by employing this optimized LSTM neural network, accurate predictions of the SOH for lithium-ion batteries are made. The results demonstrate a notable improvement in prediction accuracy by 35.71% and a significant increase in prediction speed by 35.5% when compared to traditional methods. This innovative approach proves to be effective in enhancing battery performance and longevity.

Water-to-air unmanned aerial vehicle (UAV) based rolling shutter optical camera communication (OCC) system with gated recurrent unit neural network (GRU-NN)

Underwater sensors and autonomous underwater vehicles (AUVs) are widely adopted in oceanic research activities. As the number of underwater sensors and AUVs is growing quickly, the bandwidth requirements are increasing accordingly. In this work, we put forward and demonstrate a large field-of-view (FOV) water-to-air unmanned aerial vehicle (UAV) based optical camera communication (OCC) system with gated recurrent unit neural network (GRU-NN) for the first time to the best of our knowledge. As the UAVs are embedded with complementary-metal-oxide-semiconductor (CMOS) cameras, there is no need to install OCC receivers (Rxs), reducing the deployment cost. Moreover, the large photo-sensitive area of the CMOS camera can support a large FOV OCC transmission without the need for precise optical alignment. Here, by utilizing the column matrix identification during the rolling shutter pattern decoding in the CMOS image sensor, the scintillation caused by water turbulence can be reduced. Besides, in the outdoor and windy environment, the UAV will experience significant movement caused by the wind making it very difficult to capture stable OCC frames in the CMOS image sensor. Here, we propose and demonstrate utilizing GRU-NN, which is a special realization of the recurrent neural network (RNN) with memory cells capable of learning the time-domain dependent signals. It is shown that the GRU-NN can learn effectively from successive image frames in time-domain and produce correct prediction even under the windy and unstable UAV flying environment. Experimental results reveal that the proposed GRU-NN can outperform the previous pixel-per-symbol labeling neural network (PPS-NN), and also can significantly reduce the computation time when compared with long-short-term-memory-neural-network (LSTM-NN). The proposed system can decode 4-level pulse-amplitude-modulation (PAM4) rolling shutter OCC patterns at data rates of 5.4 kbit/s and 3.0 kbit/s under clear and cloudy water, respectively, fulfilling the pre-forward error correction bit-error rate (pre-FEC BER = 3.8 × 10−3). We also demonstrate that the UAV based OCC system can support data rates of 5.4 kbit/s, 4.2 kbit/s, and 3.0 kbit/s at distances of 2.2 m, 3.2 m, and 4.2 m, respectively, at outdoor and windy environments.

Electrocardiograph analysis for risk assessment of heart failure with preserved ejection fraction: A deep learning model.

Heart failure with preserved ejection fraction (HFpEF) requires an efficient screening method. We developed a deep learning model (DLM) to screen HFpEF risk using electrocardiograms (ECGs). A cohort study was conducted utilising data from Cohorts A and B. A convolutional neural network-long short-term memory (CNN-LSTM) DLM was employed. HFpEF risk was determined by left ventricular end-diastolic pressure (LVEDP) and clinical symptoms. The DLM was trained by ECGs. LVEDP for each patient was collected through invasive left ventricular catheterisation. Cohort A and B comprised data from individuals at high risk for HFpEF (LVEDP>12mmHg) and low risk for HFpEF (LVEDP≤12mmHg). The model was trained on Cohort A and prospectively validated on Cohort B. A total of 238 patients underwent ECG and left ventricular catheterisation for model training in Cohort A, and 117 patients for validation in Cohort B. The DLM achieved 78% accuracy in assessing HFpEF risk in Cohort A, while in Cohort B, it demonstrated 78% accuracy, 71.9% specificity, and 71.7% sensitivity. In the validation Cohort B, the DLM-identified high-risk HFpEF group exhibited significantly higher prevalence of diabetes (22.03%-11.86%, P<0.01), higher BMI indices (25.92-24.22kg/cm2, P<0.01), and lower usage history of calcium channel blockers (CCB) (11.76%-28.81%, P<0.01) compared with the DLM-identified low-risk HFpEF group. Traditional HFpEF indicators, including B-type natriuretic peptide (BNP) (22-20pg/mL, P=0.71) and E/E' (8.25-8.5, P=0.66), did not exhibit significant differences between the two groups. The DLM offers an accurate, cost-effective tool for HFpEF risk assessment, potentially facilitating early detection and improved clinical management.

Photometric redshift estimation for CSST Survey with LSTM Neural Networks

Abstract Accurate estimation of photometric redshifts (photo-zs) is crucial for cosmological surveys. Various methods have been developed for this purpose, such as template fitting methods and machine learning techniques, each with its own applications, advantages, and limitations. In this study, we propose a new approach that utilizes a deep learning model based on Recurrent Neural Networks (RNN) with Long Short-Term Memory (LSTM) to predict photo-z. Unlike many existing machine learning models, our method requires only flux measurements from different observed filters as input. The model can automatically learn the complex relationships between the flux data across different wavelengths, eliminating the need for manually extracted or derived input features, thereby providing precise photo-z estimates. The effectiveness of our proposed model is evaluated using simulated data from the Chinese Space Station Telescope (CSST) sourced from the Hubble Space Telescope Advanced Camera for Surveys (HST-ACS) and the COSMOS catalog, considering anticipated instrument effects of the future CSST. Results from experiments demonstrate that our LSTM model, compared to commonly used template fitting and machine learning approaches, requires minimal input parameters and achieves high precision in photo-z estimation. For instance, when trained on the same dataset and provided only with photometric fluxes as input features, the proposed LSTM model yields one-third of the outliers fout observed with a Multi-Layer Perceptron Neural Network (MLP) model, while the normalized median absolute deviation $\rm \sigma _{NMAD}$ is only two-thirds that of the MLP model. This study presents a novel approach to accurately estimate photo-zs of galaxies using photometric data from large-scale survey projects.

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

Modeling of an inductive displacement sensor based on 1DCNN-LSTM-AT

Abstract The input-output relationship of inductive displacement sensors is typically nonlinear, which demands numerous computational resources for precise calculation. Therefore, the traditional analytical methods for the inductive displacement sensors cannot meet the real-time high-precision measurement requirements. In response to the aforementioned issues, this paper proposes an optimized Long Short-Term Memory (LSTM) neural networks based on one-dimensional convolutional neural networks (1DCNN) to modeling the input-output relationship of the inductive displacement sensor. First, the measurement principle of the inductive displacement sensors was analyzed, and the analytical model of the sensor output was derived. And the influences of the key parameters of sensors on the relationship between the induced voltage and the displacement were studied. Then, the 1DCNN-LSTM-AT network for modelling the input-output relationship of the inductive displacement sensor was studied. The spatial features of historical induced electromotive force (EMF) data generated by the induction coil were initially extracted using the 1DCNN network. And these spatial features were then utilized as input to the LSTM neural network to capture the temporal features of the historical induced EMF data. Subsequently, the spatiotemporal features of the induced EMF data were fed into the regression prediction layer to compute the displacement measurement results corresponding to the current input. Moreover, the attention mechanism was used for the 1DCNN-LSTM model to enhance the prediction accuracy and stability of the model. Finally, the experimental results demonstrate that the proposed 1DCNN-LSTM-AT model achieves an average absolute percentage error (MAPE) of 3.1%, significantly outperforming traditional models such as LSTM (29.3%), CNN (4.7%), and ANN (4.3%). This paper provides a new method for modelling the nonlinear relationships of the inductive displacement sensor, and presents a fresh perspective for research in the data processing of nonlinear sensors.

Novel state of charge estimation method of containerized Lithium–Ion battery energy storage system based on deep learning

State of charge (SOC) is a critical indicator for lithium–ion battery energy storage system. However, model-driven SOC estimation is challenging due to the coupling of internal charging and discharging processes, ion diffusion, and chemical reactions in the electrode materials. These factors result in frequent changes in their internal characteristics. Therefore, we devise a novel deep learning SOC estimation method based on the attention mechanism (A) and convolutional neural networks-long short term memory (CNN–LSTM) network model. The CNN layers are employed to extract the critical features of the battery dataset. The LSTM layers are utilized to capture the long-term dependencies in the time series data. The attention mechanism can assign attention weights to different features within the output of the convolution layer. The A–CNN–LSTM method not only amplifies the positive influence of critical knowledge but also provides a global reference. The comparative experiments of three datasets and ablation study are carried out. The root mean square error, mean absolute error, and mean absolute percentage error obtained from the A–CNN–LSTM model decreased by 32.75 %, 45.88 %, and 53.36 %, respectively, compared with the best results obtained from the existing CNN, LSTM, gate recurrent unit (GRU), CNN–LSTM, and CNN–GRU models. The proposed model demonstrates greater stability and consistency than other models, underscoring its efficacy in SOC estimation.

Fast stability enhancement of inverter‐based microgrids using NGO‐LSTM algorithm

AbstractTo improve the stability of the inverter‐based microgrid (MG), this paper employs a novel data‐driven based method to coordinately adjust control parameters of inverters in a fast local manner. During the design process, an offline eigenvalue based optimization problem that is used to calculate the optimal control parameters under various operating conditions is first constructed. In order to reduce reliance on full system information, a feature selection algorithm is utilized to extract the most relevant local measurements that influence the adjustment of each control parameter. Then, regarding local measurements as input variables and optimal control parameters as output variables, based on northern goshawk optimization (NGO) and long short‐term memory (LSTM) network, a novel deep learning algorithm is proposed to train the local parameter adjustment model (LPAM) by learning the mapping relationship between them. During the application, to guarantee the stability of MG all the time, a security region based shielding mechanism is developed, where the improper control parameter adjustment will be replaced by a safe one. The case study indicates that the proposed algorithm has better mapping accuracy than traditional LSTM neural networks and also faster calculation speed than the traditional offline optimization‐based method. The effectiveness and advantages of the proposed method are demonstrated in a modified 9‐bus MG.

Productivity forecasting of the solar still with phase change material using LSTM and GRU neural networks

Abstract Solar desalination is one of the renewable energy techniques by which freshwater can be obtained economically. Solar desalination experiments are time and resource-consuming methods; there is a need for a robust system to identify the serviceability of the solar still in a specific region. The objective of this study is to develop a forecasting model using artificial neural networks to predict freshwater productivity. The study specifically aims to compare the accuracy of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural networks in forecasting the productivity of Conventional Solar Stills (CSS) and Solar Stills with Phase Change Material (SSPCM). The current investigation involved analysing the experimental outcomes of a solar still that employed phase change material (PCM) and pin fins. Palmitic acid was implemented as the energy storage material and was placed beneath the absorber plate. The neural network model was trained and validated using time-series solar still experimental data. Different statistical measures were utilised to evaluate the accuracy of Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU). The results indicate that the freshwater productivity forecasted by LSTM exhibited greater accuracy than GRU. Specifically, the coefficient of determination values for LSTM were 0.96 and 0.98 for the CSS and SSPCM, respectively, which were better than the corresponding values for GRU.

A New Epileptic Seizure Prediction Framework Based on Electroencephalography Signals

This research seeks to evaluate how effectively seizures can be predicted and managed in epilepsy using a specialized deep learning model based on Long Short-Term Memory (LSTM) neural networks. The model leverages non-invasive scalp electroencephalography (EEG) recordings for predicting seizures. To develop and assess the proposed LSTM neural network model, a comprehensive dataset was gathered. The model emphasizes achieving high sensitivity and reducing false alarms to improve its real-time applicability. The evaluation involved various metrics to measure accuracy, sensitivity, and rates of false positives and false negatives. The effectiveness of the proposed LSTM neural network model was outstanding, with accuracy rates ranging from 99.07% to 99.95%. Notably, the sensitivity score of 1 confirmed precise prediction for all seizure cases. The model demonstrated minimal false positive and false negative rates, highlighting its reliability in predicting seizures. This study emphasizes the promising potential of the proposed LSTM neural network model in providing advanced warning for seizures. The high accuracy and sensitivity rates suggest its usefulness in enabling timely preventive measures for patients, ultimately reducing the occurrence of seizures. This innovative approach holds significance in enhancing the overall management and quality of life for individuals dealing with epilepsy.

Soft sensor modeling method and application based on TSECIT2FNN-LSTM

To address the issue of low accuracy in soft sensor modeling of key variables caused by multi-variable coupling and parameter sensitivity in complex processes, this paper introduces a TSK-type-based self-evolving compensatory interval type-2 fuzzy Long short-term memory (LSTM) neural network (TSECIT2FNN-LSTM) soft sensor model. The proposed TSECIT2FNN-LSTM integrates the LSTM neural network with the interval type-2 fuzzy inference system to address long-term dependencies in sequence data by utilizing the gate mechanism of the LSTM neural network. The TSECIT2FNN-LSTM structure learning algorithm uses the firing strength of the network rule antecedent to decide whether to generate new rules to improve the rationality of the network structure. TSECIT2FNN-LSTM parameter learning utilizes the gradient descent method to optimize network parameters. However, unlike other interval type-2 fuzzy neural network gradient calculation processes, the error term in the LSTM node parameter gradient of TSECIT2FNN-LSTM is propagated backwards in the time dimension. Additionally, the error term is simultaneously transferred to the upper layer network to enhance network prediction accuracy and memory capabilities. The TSECIT2FNN-LSTM soft sensor model is utilized to predict the alcohol concentration in wine and the nitrogen oxide emission in gas turbines. Experimental results demonstrate that the proposed TSECIT2FNN-LSTM soft sensing model achieves higher prediction accuracy compared to other models.

Research on risk detection and management based on machine learning

With the rapid advancement in financial risk management, machine learning techniques are playing an increasingly crucial role in credit assessment and risk detection. They provide financial institutions with more scientific and precise platforms and methodologies for risk management. This paper explores various traditional machine learning methods in risk assessment, introduces the XGBoost ensemble learning method, and integrates traditional machine learning with Long Short-Term Memory (LSTM) neural networks to enhance the generalization capability of risk control and credit assessment models. This approach offers new perspectives and improvement directions for risk assessment standards and practices in the financial industry, affirming the potential application of machine learning technologies in future risk management.The results show that the random forest and decision tree have excellent accuracy and recall rates for distinguishing fraudulent transactions. After the introduction of LSTM neural networks, the accuracy and recall rates of fraudulent transaction recognition models reach around 99%, indicating that these models have good adaptability for fraud risk recognition.

Empowering Digital Civility with an NLP Approach for Detecting Twitter Cyberbullying through Boosted Ensembles

As the number of social networking sites grows, so do cyber dangers. Cyberbullying is harmful behaviour that uses technology to intimidate, harass or harm someone, often on social media platforms like Twitter. Machine learning is the optimal approach for cyberbullying detection on Twitter to process large amounts of data, identify patterns of offensive behaviour, and automate the detection process for corpus of tweets. To identify cyber threats using a trained model, the Boosted Ensemble (BE) technique is assessed with various machine learning algorithms such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Naive Bayes (NB), Decision Tree (DT), Support Vector Machine (SVM), Bidirectional LSTM (BILSTM), Recurrent Neural Network LSTM (RNN-LSTM), Multi-Modal Cyberbullying Detection (MMCD) and Random Forest (RF). These classifiers are trained on the vectorized data to classify the tweets to identify cyberbullying threats. The proposed framework can detect cyberbullying cases precisely on tweets. The significance of the work lies in detecting and mitigating cyber threats in real-time and it impacts in enhancing the safety and well-being of social media users by reducing instances of cyberbullying and other cyber threat. The comparative analysis is done using metrices like accuracy, precision, recall and F1-score and the comparison results shows that boosted ensemble technique outperforms other compared algorithms with its overall performance. The accuracy rate of CNN, LSTM, NB, DT, SVM, RF, BILSTM and BE (92.5%, 93.5%, 84.6%, 88%, 89.3%, 92%, 93.75% and 96%), precision rate of CNN, LSTM, NB, DT, SVM, RF, RNN-LSTM and BE (90.2%, 91.3%, 88%, 85%, 86%, 91.6%, 92.1% and 94%), recall rate of CNN, LSTM, NB, DT, SVM, RF, BILSTM and BE (89.8%, 90.7%, 90%, 82%, 88.67%, 89%, 91.04% and 93.7%) and F1 score of CNN, LSTM, NB, DT, SVM, RF, MMCD and BE (90.6%, 91.8%, 85%, 84.56% 87.2%, 90%, 84.6% and 94.89%).

Condition assessment and predictive maintenance for contact probe using health index and encoder‐decoder LSTM model

AbstractContact probe is broadly used for the continuous monitoring of microelectronic components in manufacturing industries. False rejection of fine product due to defective contact probe significantly reduces the yield in production. Traditionally, defect detection for contact probes heavily depends on a valid range manually defined by engineers over the measured value of certain parameters. However, the subjective range defined according to engineer experience is prone to trigger a high rate of false alarms due to the inherent noise in the measured parameters. To address this issue, we construct a health index (HI) with the contact resistance‐directly‐related features to help monitor and assess the condition of contact probe. Based on the established HI, we develop Long Short‐Term Memory (LSTM) encoder‐decoder machine learning model to assess the condition of contact probe by forecasting the HI value in the future. Encoders from LSTM and convolutional neural network (CNN) are selected as the encoder‐decoder architecture for the sequence‐to‐sequence prediction due to their advantage in extracting the correlation of features at different scales. An explainable Artificial Intelligence (XAI) technique named Local interpretable model‐agnostic explanations (LIME) is used to quantify the contribution of each feature to the model prediction. The encoder from CNN is found to outperform the LSTM encoder in extracting the inter‐feature correlation. Finally, the predicted HI is used to signal the alarm for the maintenance action of contact probe when its value is below a predefined threshold. Comparison between the action alarm triggered by the developed HI and the actual maintenance records suggests that the proposed approach achieves at least 75% accuracy for the triggered alarm in the next 15 mins.

Evaporation temperature prediction of the refrigerant-direct convective-radiant cooling system based on LSTM neural network

The existing radiant cooling systems are effective at handling indoor sensible heat load. However, the indoor latent load needs to be treated with an additional dehumidification system, which will increase the complexity and initial investment of the system. To settle this dilemma, a novel aluminum column-wing type refrigerant-direct convective-radiant cooling (ACT-RCC) system was proposed, which adopted refrigerant as direct medium and provided cooling and independent dehumidification simultaneously. Experiments were conducted in an enthalpy difference laboratory with the indoor air temperature of 22.0 °C − 30.0 °C and relative humidity of 60.0%. A numerical model of the ACT-RCC terminal was established and validated with the experimental data. To meet the indoor heat and latent load of residential buildings, a heuristic approach based on Long Short-Term Memory (LSTM) neural network for predicting the evaporation temperature of this system was proposed. Results showed that the predicted values were accurate enough by measured data and the maximum deviation of the evaporation temperature was 0.7 °C. The relationship between the evaporation temperature, indoor air temperature and relative humidity, and heat moisture ratio was derived. Besides, the indoor thermal comfort of the selected bedroom was also evaluated. Results showed that when the indoor air temperature and relative humidity remained 26.0 °C and 40.0% − 70.0%, the thermal comfort of this bedroom can achieve Level Ⅰ by adjusting the evaporation temperature of the ACT-RCC system of 2.0 °C − 19.8 °C.

Adaptive Neural Control for Hysteretic Nonlinear Systems With Hysteresis Neural Direct Inverse Compensator and Its Application.

Aiming at high precision control for a class of hysteretic nonlinear systems, a new hysteresis direct inverse compensator-based adaptive output feedback control scheme is designed in this article. First, a novel long short-term memory neural network (LSTMNN)-based hysteresis inverse compensator is established to compensate the asymmetric hysteresis nonlinearity, where the LSTMNN is used as the prediction mechanism for model operator weights, rather than the overall mapping of hysteresis input and output. Second, by designing the modified high-gain K-Filter states observer and the error transformed function, the unmeasurable states are estimated with arbitrarily small estimation error and the prespecified tracking performance is achieved. Lastly, the biconical dielectric elastomer actuator (DEA) motion platform is constructed. Then, the effectiveness of the proposed LSTMNN-based hysteresis inverse compensator and control scheme are verified on the experimental platform. The experimental results illustrate the effectiveness and advantages of proposed control scheme.

Mpox outbreak: Time series analysis with multifractal and deep learning network.

This article presents an overview of an mpox epidemiological situation in the most affected regions-Africa, Americas, and Europe-tailoring fractal interpolation for pre-processing the mpox cases. This keen analysis has highlighted the irregular and fractal patterns in the trend of mpox transmission. During the current scenario of public health emergency of international concern due to an mpox outbreak, an additional significance of this article is the interpretation of mpox spread in light of multifractality. The self-similar measure, namely, the multifractal measure, is utilized to explore the heterogeneity in the mpox cases. Moreover, a bidirectional long-short term memory neural network has been employed to forecast the future mpox spread to alert the outbreak as it seems to be a silent symptom for global epidemic.

Compensation Algorithm for Wind Turbine Main Shaft Loads Based on LSTM

Abstract For wind turbine development, load size is the core and critical factor. Currently, most companies employ Bladed software for load calculations, which enjoys widespread use in engineering applications due to its efficiency and ease of use. However, constrained by the limitations of blade element theory and momentum theory, there still exist discrepancies between the simulation results and actual conditions. This paper aims to combine simulation algorithms with the application of an LSTM (Long Short-Term Memory) neural network to develop a compensation algorithm for the main shaft loads of wind turbines. By doing so, it endeavors to realize real-time monitoring of the main shaft loads, thereby providing a foundation for achieving high-precision control in wind turbine systems.

Dynamic Prediction for Pollutant Emissions of Coal-fired Power Plant Based on CNN-LSTM-Attention

Abstract The prediction accuracy of pollutant emissions by using traditional modeling methods is unsatisfactory in dynamic conditions. To overcome the problem, data-driven modeling was introduced to build the dynamic model of pollutant emissions of power plants in this paper. Combining with the running data of a 300MW circulating fluidized bed (CFB) unit, the dynamic prediction models of SO2 and NO x emissions were established respectively by using conventional neural network-long short-term memory and attention mechanism (CNN-LSTM-Attention). Moreover, LSSVM, LSTM and CNN-LSTM were introduced for comparison to demonstrate the superiority of CNN-LSTM-Attention model respectively. Simulation results indicate that model can imitate change trend of actual data with high accuracy over a long period of time. Compared with LSSVM, LSTM and CNN-LSTM, the proposed model has better modeling performance under different load conditions. This work provides certain guidance for the application of deep learning in the industrial field.