Long Short-term Memory Neural Network Research Articles

Volterra-Aided Neural Network Equalization for Channel Impairment Compensation in Visible Light Communication System

This paper addresses the channel impairment to enhance the system performance of visible light communication (VLC). Inspired by the model-solving procedure in the conventional equalizer, the channel impairment compensation is formulated as a spatial memory pattern prediction problem, then we propose efficient deep-learning (DL)-based nonlinear post-equalization, combining the Volterra-aided convolutional neural network (CNN) and long-short term memory (LSTM) neural network, to mitigate the system nonlinearity and then recover the original transmitted signal from the distorted one at the receiver end. The Volterra structure is employed to construct a spatial pattern that can be easily interpreted by the proposed scheme. Then, we take advantage of the CNN to extract the implicit feature of channel impairments and utilize the LSTM to predict the memory sequence. Results demonstrate that the proposed scheme can provide a fairly fast convergence during the training stage and can effectively mitigate the overall nonlinearity of the system at testing. Furthermore, it can recover the original signal accurately and exhibits an excellent bit error rate performance as compared with the conventional equalizer, demonstrating the prospect and validity of this methodology for channel impairment compensation.

Implementation and Field Test of Optimal Pump Scheduling in the Multiproduct Refined Oil Transmission System

Optimal pump scheduling ensures the safe and economic operation of the oil pipelines, which is also of vital significance for energy conservation as well as carbon emission reduction. This article proposes a framework adapting the data-driven pressure loss estimation with the long short-term memory neural network (LSTM-NN) and pump characteristics fitting with quadratic polynomials to model-based optimal pump scheduling for the multiproduct refined oil transmission system. The data-driven methods are data-adaptive with periodical field data in a rolling-training manner, and thus can reflect actual working conditions in the transmission pipelines. The presented LSTM-NN for pressure loss estimation integrating the characteristics of multiproduct refined oil pipeline transmission has good performance with an acceptable mean squared error of 0.016 MPa <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> in real applications. The optimization model makes the time precision down to the minute level, which is more convenient for the system dispatchers to operate the pumps following the optimal schedule. Practical implementation and field test are carried out in a real-world pipeline system, which verifies that the proposed framework is more practical and better-performed than the manual schedule.

Air Conditioning Load Prediction of an Office Building Based on Long Short Term Memory Neural Network

Background: Energy conservation has always been a major issue in our country, and the air conditioning energy consumption of buildings accounts for the majority of the energy consumption of buildings. If the building load can be predicted and the air conditioning equipment can respond in advance, it can not only save energy but also extend the life of the equipment. Introduction: The neural network proposed in this paper can deeply analyze the load characteristics through three gate structures, which helps improving the prediction accuracy. Combined with the grey relational degree method, the prediction speed can be accelerated. Method: This paper introduces a grey relational degree method to analyze the factors related to air conditioning load and selects the best ones. A Long Short-Term Memory Neural Network (LSTMNN) prediction model was established. In this paper, grey relational analysis and LSTMNN are combined to predict the air conditioning load of an office building, and the predicted results are compared with the real values. Results: Compared with the Back Propagation Neural Network (BPNN) prediction model and Support Vector Machine (SVM) prediction model, the simulation results show that this method has a better effect on air conditioning load prediction. Conclusion: Grey relational degree analysis can extract the main factors from the numerous data, which is more convenient and quicker without repeated trial and error. LSTMNN prediction model not only considers the relation of air conditioning load on time series but also considers the nonlinear relation between load and other factors. This model has higher prediction accuracy, shorter prediction time, and great application potential.

Trip Pricing Scheme for Electric Vehicle Sharing Network With Demand Prediction

With promising benefits such as emission reduction, traffic congestion alleviation and parking space saving, electric vehicle sharing systems have attracted increasing attentions. This paper proposes a Trip Pricing Scheme (TPS) for a large-scale EV-sharing network with Shared Electric Vehicle (SEV) demand prediction. In the proposed system, the SEV traffic demand is firstly predicted through a model which includes a cascade graph convolutional neural network and a long-short term memory neural network. Based on this, the TPS is modelled as a mixed-integer nonlinear programming problem, which aims at maximizing the total system profit from EV sharing business. The proposed TPS determines the optimal combination of the two price adjustment levels, which provides incentives to the spatial-temporal distribution of SEVs’ traffic flows to maximize the system’s profit. Numerical simulations are conducted to validate the effectiveness of the proposed method.

Short-term wind speed forecasting based on two-stage preprocessing method, sparrow search algorithm and long short-term memory neural network

Wind energy, as an environment-friendly and renewable energy source, has become one of the most effective alternatives to conventional power sources. However, the intermittent nature of wind speed and the interference of noise signal bring several challenges to the safety and reliability of power grid operation. To tackle this issue, a two-stage preprocessing strategy is designed, and the short-term wind speed prediction model based on long short-term memory (LSTM) is proposed. Firstly, singular spectrum analysis (SSA) is introduced to extract the target data and filter the noise data. Next, the denoised sequence is decomposed by variational mode decomposition (VMD) into multiple intrinsic mode functions (IMFs), which are further aggregated by sample entropy (SE). Besides, the hyper-parameters of LSTM neural network are optimized by the newly sparrow search algorithm (SPSA) possessing excellent global optimization ability. Subsequently, the aggregated sequences are coupled with the SPSA-LSTM modules synchronously. The ultimate wind speed forecasting results are obtained by superimposing the predicted values of all sequences. In order to evaluate the effectiveness of proposed approach, two case studies are conducted based on two datasets collected from different sites with 10-min and 1-hour intervals by comparing seven relevant models. The experimental results demonstrate that the proposed SSA-VMD-SE-SPSA-LSTM can adequately extract the inherent features of wind speed series, thus achieving higher prediction accuracy.

A Novel Technique for Data Quality Improvement in Human Health PHM

Human healthcare data cover different signals related to the functioning of the human body such as blood pressure, blood glucose, heart rate, among others. This kind of data plays an important role on ill patients in the intensive care unit. Unfortunately, the recorded data may include connection &amp; human errors, measurement errors due to the movement of patients, among others. These errors are better known as Artifacts and they should be removed in case the data needs to be used for clinical purposes. Different methods have been proposed for artifact detection; however, the existing methods only analyze one signal at a time or rely on feature engineering. In this study, we present an alternative solution for artifact detection that integrates signals such as Intracranial Pressure (ICP), Electrocardiogram (ECG), and Arterial Blood Pressure (ABP). Time raw domain data, Fourier transform, and the combination of both were studied as the input of neural network models. Four deep learning algorithms were employed: convolutional neural network (CNN), long short-term memory (LSTM), bidirectional LSTM (BiLSTM), and transformer neural network. By performing a cross-validation ensemble using a dataset of 39 patients with noisy signals, the Fourier transform and CNN (FT-CNN) model outperformed the other deep learning models in terms of accuracy and computational time. This study shows that deep learning, Fourier transform, and cross-validation ensemble combined have a great potential in data quality improvement in healthcare.

Development of a Coupled EnergyPlus-MATLAB Simulation Based on LSTM for Predictive Control of HVAC System

To avoid unnecessary energy waste due to the single temperature setpoints of the heating, ventilation, and air conditioning (HVAC) system during the seasonal variation period, this study proposed a long-short-term memory (LSTM) neural network to predict and control the temperature setpoint. The thermal comfort, cooling rate, and heating rate were predicted with outdoor environment parameters and occupant count. A large scale of operation data was collected from the EnergyPlus simulation, which was previously developed based on the characteristics of a real case study house. Different kinds of input characteristics were offered to test the stable use of LSTM and other artificial neural networks. This paper discusses the development of a Matlab EnergyPlus co-simulation to predict and control the temperature setpoints of a variable air volume system, especially the relationship between temperature setpoint and energy consumption. The simulation results indicate the advantages of the LSTM prediction of energy consumption and the potential for energy saving with predictive control.

Long Short Term Memory Neural Network (LSTMNN) and inter-symbol feature extraction for 160 Gbit/s PAM4 from silicon micro-ring transmitter

We propose and experimentally illustrate the use of Long Short Term Memory Neural Network (LSTMNN) and inter-symbol feature extraction (IFE) to recover 4-level pulse amplitude modulation (PAM4) data transmitted from a silicon micro-ring modulator (SiMRM) operated at 160 Gbit/s. Pre-hard-decision (HD) forward error correction (FEC) requirement (i.e. bit-error-rate, BER < 3.8 × 10−3) is achieved in the 160 Gbit/s PAM4 signal produced using a 47-GHz bandwidth SiMRM after 1 km single-mode-fiber (SMF) transmission.

Method of Hydrological Time Series Recovery Using Neural Network Technologies

n this work we applied machine learning methods in the field of hydrological measurements to recover (to predict) missing or damaged data. Field measurements of temperature field that were taken as a typical example, were carried out on the Sea of Japan shelf (the Peter the Great Bay) in October 2021 using a vertical moored thermostring. Recording of one of its sensors was partly deleted in manual way. To recover the missing part, we used the method of averaging the readings of the nearest sensors, as well as one-dimensional and two-dimensional Long Short-Term Memory (LSTM) neural network models. The results were compared with real data; recovery quality was assessed using the Root Mean Square Error (RMSE) of the prediction. The study showed that two-dimensional LSTM provides more accurate data recovering, with a minimum RMSE of 0.014 ± 0.006. The simulation results prove that multidimensional recovering can significantly increase the length of the predicted series while maintaining acceptable accuracy and compensate for the error accumulation effect that is observed with one-dimensional recovering. We believe that multidimensional data recovery method based on the LSTM neural network is very promising for non-stationary time series.

Regression model-based hourly aggregated electricity demand prediction

The ability to predict ggregated electricity demand of n electrical grid on an hourly basis is crucial for energy and demand management. In this study, demand and its categorical features data for three years are segregated into four seasons and then fed to an efficient Machine Learning Categorical Boosting (ML CatBoost) Regressor model to predict the next year’s demand. It uses a new gradient boosting algorithm that handles categorical features adeptly. Also, this model uses a new scheme for estimating leaf values while choosing tree structure which reduces overfitting. Further, hourly electricity demand data from the Electricity Reliability Council of Texas (ERCOT western) is used as the benchmark data to evaluate the CatBoost model. Moreover, five other ML models were developed, analyzed, and tested for the same ERCOT data for predicting the hourly aggregated electricity demand. The suggested model is compared with the long short-term memory neural network (LSTM-NN) and five other ML models in terms of performance evaluation matrices. Here, one additional performance evaluation parameter, the Coefficient of variation Root Mean Squared (CV-RMSE) is evaluated in addition to the benchmark paper’s parameters. In addition, the importance of accurate prediction in the electric grid for clean energy is discussed.

Prediction of shear stress induced by shoaling internal solitary waves based on machine learning method

Recently, the interactions between internal solitary waves (ISWs) and the seabed have directed increasing attention to ocean engineering and offshore energy. In particular, ISWs induce bottom currents and pressure fluctuations in deep water. In this paper, we propose a method for predicting the shear stress induced by shoaling ISWs based on machine learning, and the developed approach can be used to quickly determine the safety and stability of ocean engineering. First, we provided a basic dataset for model training. Four machine learning models were selected to predict the shear stress induced by shoaling ISWs under different trim conditions. The results indicated that the performance of the convolutional neural network-long short-term memory (CNN-LSTM) forest prediction model was significantly better than the three other tested models, including long short-term memory (LSTM), support vector regression (SVR) and deep neural network (DNN) models. Therefore, the CNN-LSTM forest prediction model was the optimal model for predicting the shear stress induced by shoaling ISWs. Specifically, each metric of the CNN-LSTM model was smaller than that of the other three, and the root mean squared error to the standard deviation ratio was closest to 0.7. In addition, the CNN-LSTM model significantly outperformed the SVR and DNN models in terms of the length of prediction time. The predicted values by the CNN-LSTM model were consistent with the experimental values. The method for predicting shear stress based on machine learning in this paper can be used to predict the shear stress induced by shoaling ISWs, guide future field experiment designs, reduce damage to the seabed caused by ISWs, and promote the development of ocean engineering in deep water.

Physics-Informed Machine Learning for Degradation Modeling of an Electro-Hydrostatic Actuator System

Machine learning (ML) methods are becoming popular in prognostics and health management (PHM) of engineering systems due to the recent advances of sensor technology and the prevalent use of artificial neural networks. In practice, mechatronic systems are by nature, prone to degradation/failure due to complex failure mechanisms and other unknown causes. As a result, degradation modeling and prediction of mechatronic systems are quite challenging especially when highly integrative and special operational conditions are considered. To overcome such challenges, artificial neural networks can be employed. This paper proposes the use of a long short-term memory (LSTM)-based multi-input neural network for degradation modeling and prediction of an Electro-Hydrostatic Actuator (EHA) system. The failure mechanisms of the EHA system are explored first, and the obtained physics-of-failure information is utilized in constructing the LSTM neural network to enhance the prediction capability of the model. An actual dataset collected from an EHA test bench is utilized to illustrate the effectiveness of the proposed physics-informed LSTM method for modeling the EHA system's degradation behavior. The result shows that the proposed method provides more accurate life prediction than several benchmark methods for the EHA system.

Active Noise Reduction with Filtered Least-Mean-Square Algorithm Improved by Long Short-Term Memory Models for Radiation Noise of Diesel Engine

This study presents an active noise control (ANC) algorithm using long short-term memory (LSTM) layers as a type of recurrent neural network. The filtered least-mean-square (FxLMS) algorithm is a widely used ANC algorithm, where the noise in a target area is reduced through a control signal generated from an adaptive filter. Artificial intelligence can enhance the reduction performance of ANC for specific applications. An LSTM is an artificial neural network for recognizing patterns in arbitrarily long sequence data. In this study, an ANC controller consisting of LSTM layers based on deep neural networks was designed for predicting a reference noise signal, which was used to generate the control signal to minimize the noise residue. The structure of the LSTM neural networks and procedure for training the LSTM controller for the ANC were determined. Simulations were conducted to compare the convergence time and performances of the ANC with the LSTM controller and those with a conventional FxLMS algorithm. The noise source adopted sounds from a single-cylinder diesel engine, while reference noises selected were single harmonics, superposed harmonics, and impulsive signals generated from the diesel engine. The characteristics of each algorithm were examined through a Fourier transform analysis of the ANC results. The simulation results demonstrated that the proposed ANC method with LSTM layers showed outstanding noise reduction capabilities in narrowband, broadband, and impulsive noise environments, without high computational cost and complexity relative to the conventional FxLMS algorithm.

Deep learning for laboratory earthquake prediction and autoregressive forecasting of fault zone stress

Earthquake forecasting and prediction have long and in some cases sordid histories but recent work has rekindled interest based on advances in early warning, hazard assessment for induced seismicity and successful prediction of laboratory earthquakes. In the lab, frictional stick-slip events provide an analog for earthquakes and the seismic cycle. Labquakes are also ideal targets for machine learning (ML) because they can be produced in long sequences under controlled conditions. Indeed, recent works show that ML can predict several aspects of labquakes using fault zone acoustic emissions (AE). Here, we extend these works with: 1) deep learning (DL) methods for labquake prediction, 2) by introducing an autoregressive (AR) forecasting DL method to predict fault zone shear stress, and 3) by expanding the range of lab fault zones studied. The AR methods allow forecasting stress at future times via iterative predictions using previous measurements. Our DL methods outperform existing ML models and can predict based on limited training. We also explore forecasts beyond a single seismic cycle for aperiodic failure. We describe significant improvements to existing methods of labquake prediction and demonstrate: 1) that DL models based on Long-Short Term Memory and Convolution Neural Networks predict labquakes under conditions including pre-seismic creep, aperiodic events and alternating slow/fast events and 2) that fault zone stress can be predicted with fidelity, confirming that acoustic energy is a fingerprint of fault zone stress. Our DL methods predict time to start of failure (TTsF) and time to the end of Failure (TTeF) for labquakes. Interestingly, TTeF is successfully predicted in all seismic cycles, while the TTsF prediction varies with the amount of preseismic fault creep. We report AR methods to forecast the evolution of fault stress using three sequence modelling frameworks: LSTM, Temporal Convolution Network and Transformer Network. AR forecasting is distinct from existing predictive models, which predict only a target variable at a specific time. The results for forecasting beyond a single seismic cycle are limited but encouraging. Our ML/DL models outperform the state-of-the-art and our autoregressive model represents a novel framework that could enhance current methods of earthquake forecasting.

A solution for co-frequency and low SNR problems in heart rate estimation based on photoplethysmography signals.

In order to realize high-accuracy heart rate (HR) estimation based on photoplethysmography (PPG) under the scenes of low signal-to-noise ratio (SNR) and co-frequency caused by motion artifacts (MAs), this paper presents a novel framework integrating two-stage variational mode decomposition (VMD) denoising method, noise compensation technology, and hidden Markov model (HMM)-based tracking algorithm. The two-stage VMD denoising method is designed to separate the HR signal from MA under low SNR scene. The noise compensation technology is applied to solve the problem of co-frequency. HMM-based HR tracking method is adopted to obtain the global optimization performance of HR estimation. The effectiveness and superiority of the proposed framework in solving problems of low SNR and co-frequency associated with motion artifacts have been verified by the HR estimation experiments carried out on three public high-SNR PPG databases (ISPC, BAMI I, BAMI II) and a self-built low-SNR database (WeData). Compared with the two classical frameworks namely joint sparse spectrum reconstruction (JOSS) and convolutional neural network-long short-term memory network (CNN-LSTM), the proposed framework obtains the lowest HR estimation errors (0.94 beats per minute (BPM) and 1.81 BPM respectively) on both BAMI 2 with the highest SNR (0.40dB) and WeData with the lowest SNR (- 9.07dB). For the low-SNR database Wedata, the average absolute error (AAE) decreases by more than 21 BPM. The research result of this study provides a solution for the realization of high-accuracy PPG-based HR estimation in exercise scenarios.

Prediction of Whole-Body Velocity and Direction From Local Leg Joint Movements in Insect Walking via LSTM Neural Networks

Extracting motion information from videos is important for quantifying data from behavioral experiments to deepen the understanding of generation mechanisms of animal behavior. For insect walking, inter-leg coordination plays a crucial role, and the thorax-coxa (ThC) and femur-tibia (FTi) joint motions of six legs reflect the walking velocity and direction. This suggests that joint-motion information based on a continuous time series is beneficial for dynamic behavior prediction. Since pose estimation from markerless videos has been extensively studied, the joint angle can be calculated accurately from videos via deep-learning algorithms such as DeepLabCut. Herein, we propose a method for the single-step and multi-step prediction of whole-body velocity and direction using leg joint angles. The method constructs models using long short-term memory (LSTM) and Hammerstein LSTM (HLSTM), with joint data as input, to predict the whole-body velocity and direction of insect walking. We investigated motion prediction with ThC and FTi joint angles using LSTM and HLSTM. The trained models predicted single-step motion information with the accuracy in the range of 73.28%–92.12% for velocity and 66.66%–87.46% for direction. Multi-step prediction for the next 10 steps showed the accuracy in the range of 99.56%–99.99% for velocity and 99.43%–99.95% for direction.

Multivariable sales prediction for filling stations via GA improved BiLSTM

Accurate sales prediction in filling stations is the basis to fill in the refined oil in time and avoid the out-of-stock as much as possible. Considering the defect of great “lag” in the general time series model, this paper summarizes the multiple factors that influence the oil sales and develops a multivariable long short-term memory (LSTM) neural network, with the hyper-parameters being improved by the genetic algorithm (GA). To further improve the prediction accuracy, the proposed LSTM neural network is generalized to bidirectional LSTM (BiLSTM), in which the impact of future factors on present sales can be taken into account by backward training. Finally, different LSTM structures and genetic algorithm parameters are tested to discuss their impact on prediction accuracy. Results demonstrated that genetic algorithm improved BiLSTM model is superior to extreme gradient boosting, ARIMA, and artificial neural network, having the highest accuracy of 89%.

Early Stator Fault Detection and Condition Identification in Induction Motor Using Novel Deep Network

Fault diagnosis of induction motor is an important task for both the researchers and industry. The interturn fault is one of the frequented faults and accounts for more than 37% of the failures. Thus, the diagnosis of interturn fault at an early stage has become vital to avoid the catastrophic failures and production losses. In this work, the advantages of deep learning-based methods are explored to detect the interturn fault at an incipient stage. Hybrid architectures has been proposed in this work for incipient inter turn fault diagnosis [i.e., 1D convolution neural network-long short-term memory (1DCNN-LSTM) and 1DCNN-gated recurrent unit (GRU) based methods]. The performance of the hybrid methods has been compared with standalone techniques (1DCNN, LSTM, and GRU), and the results show that the hybrid models (1DCNN-LSTM and 1DCNN-GRU) outperform the individual architectures (in terms of accuracy, sensitivity, and specificity), for fault diagnosis and isolation. The computational time has also been found to be comparable, which is an added advantage for fast diagnosis tasks. The approach combines the feature extraction and classification tasks, to perform early diagnosis, in presence of ambiguous conditions. Here, the incipient diagnosis of fault has been augmented with condition identification i.e., healthy (balanced voltages), healthy (unbalanced voltages), faulty (balanced voltages), and faulty (unbalanced voltages), to effect both early detection and isolation. The performance metrics show the robustness of the proposed method for not only detecting fault, but also identifying such confusing conditions for critical applications, in which induction motors are employed.

Credit Card Fraud Detection Using LSTM Algorithm

With the rapid growth of consumer credit and the huge amount of financial data developing effective credit scoring models is very crucial. Researchers have developed complex credit scoring models using statistical and artificial intelligence (AI) techniques to help banks and financial institutions to support their financial decisions. Neural networks are considered as a mostly wide used technique in finance and business applications. Thus, the main aim of this search is to help bank management in scoring credit card clients using machine learning by modelling and predicting the consumer behavior with respect to two aspects: the probability of single and consecutive missed payments for credit card customers. The proposed model is based on the bidirectional Long-Short Term Memory (LSTM) model to give the probability of a missed payment during the next month for each customer. The model was trained on a real credit card dataset and the customer behavioral scores are analyzed using classical measures such as accuracy, Area Under the Curve, Brier score, Kolmogorov–Smirnov test, and H-measure. Calibration analysis of the LSTM model scores showed that they can be considered as probabilities of missed payments. The LSTM model was compared to four traditional machine learning algorithms: support vector machine, random forest, multi-layer perceptron neural network, and logistic regression. Experimental results show that, compared with traditional methods, the consumer credit scoring method based on the LSTM neural network has significantly improved consumer credit scoring.

Predicting Blood Glucose in Type-1 Diabetic Patients using Deep Learning Techniques - Approaches and Challenges

Diabetes Mellitus (DM) is a metabolic disorder where the body fails to produce the digestive hormone insulin, or the body’s ability to respond to insulin is limited. This situation leads to abnormal metabolism of carbohydrates and elevated blood sugar level. Type-1 diabetes (T1DM) is a condition arises mainly due to auto immunity disorder in which the immunity cells of the body mistakenly destroy the beta cells in the pancreas, which produce insulin. T1DM patients require to have proper control of their blood glucose level through proper medication, physical activities and continuous monitoring of blood glucose levels. A wide range of advanced wellbeing innovations, particularly computerized applications, have been growing quickly to assist individuals with dealing with their diabetes. Artificial Intelligence is a rapidly growing field, and its applications to diabetes research are becoming significantly more quickly. This paper is a review of six studies of existing neural networkbased models for the prediction of future blood glucose level in T1DM patients and describes some challenges to predict future blood glucose with the available data. These models include prediction of blood glucose level using Convolutional Neural Networks (CNN), Feedforward Neural Network (FNN), Recurrent Neural Network (RNN) implemented using Long Short Term Memory (LSTM), Convolutional Recurrent Neural Network (CRNN), Bidirectional LSTM (BiLSTM) and Dilated Recurrent Neural Networks (DRNN)