Bidirectional Long Short-term Memory Neural Network Research Articles

Phishing Detection in Blockchain Transaction Networks Using Ensemble Learning

The recent progress in blockchain and wireless communication infrastructures has paved the way for creating blockchain-based systems that protect data integrity and enable secure information sharing. Despite these advancements, concerns regarding security and privacy continue to impede the widespread adoption of blockchain technology, especially when sharing sensitive data. Specific security attacks against blockchains, such as data poisoning attacks, privacy leaks, and a single point of failure, must be addressed to develop efficient blockchain-supported IT infrastructures. This study proposes the use of deep learning methods, including Long Short-Term Memory (LSTM), Bi-directional LSTM (Bi-LSTM), and convolutional neural network LSTM (CNN-LSTM), to detect phishing attacks in a blockchain transaction network. These methods were evaluated on a dataset comprising malicious and benign addresses from the Ethereum blockchain dark list and whitelist dataset, and the results showed an accuracy of 99.72%.

Simulated annealing algorithm optimized GRU neural network for urban rainfall-inundation prediction

Abstract In the context of global climate change and the continuous development of urban areas, rainfall-inundation modeling is a common approach that provides critical support for the protection and early warning of urban waterlogging protection. The present study conducts a data-driven model for hourly urban rainfall-inundation depth prediction, which is based on a gated recurrent unit (GRU) neural network and uses the simulated annealing (SA) algorithm for the hyperparameter optimization of GRU, namely the SA-GRU model. To verify the performance of the proposed model, backpropagation, long short-term memory (LSTM), and bidirectional LSTM (BiLSTM) neural networks are set as benchmarks. Results show that the SA-GRU has high accuracy in the case of short-term inundation prediction, with the Nash–Sutcliffe efficiency from 0.999 to 0.596 for the 1-h-ahead to 8-h-ahead predictions. And further research reveals that the SA-GRU integrates the significant optimization of SA, with an average 20% reduction of the root mean square error within the first eight prediction periods, and the efficient training speed of GRU, with 23.7% faster than LSTM and 44.2% faster than BiLSTM. In conclusion, the SA-GRU excels in urban inundation prediction, demonstrating its value in flood management and decision-making.

Track Signal Intrusion Detection Method Based on Deep Learning in Cloud-Edge Collaborative Computing Environment

Aiming at the low accuracy of the track signal intrusion detection (IDe) algorithm in the traditional cloud-side collaborative computing environment, this paper proposes a deep learning (D-L)-based track signal IDe method in the cloud edge collaborative computing environment. First, the main framework of the IDe method is constructed by comprehensively considering the backbone network, network transmission and ground equipment, and edge computing (EC) is introduced to cloud services. Then, the The CNN (Convolutional Neural Networks)-attention-based BiLSTM (Bi-directional Long Short-Term Memory) neural network is used in the cloud center layer of the system to train the historical data, a D-L method is proposed. Finally, a pooling layer and a dropout layer are introduced into the model to effectively prevent the overfitting of the model and achieve accurate detection of track signal intrusion. The purpose of introducing the pooling layer is to accelerate the model convergence, remove the redundancy and reduce the feature dimension, and the purpose of introducing the dropout layer is to prevent the overfitting of the model. Through simulation experiments, the proposed IDe method and the other three methods are compared and analyzed under the same conditions. The results show that the F1 value of the method proposed in this paper is optimal under four different types of sample data. The F1 value is the lowest of 0.948 and the highest of 0.963. The performance of the algorithm is better than the other three comparison algorithms. The method proposed in this paper is important for solving the IDe signal in the cloud-edge cooperative environment, and also provides a theoretical basis for tracking the signal IDe direction.

RF-BiLSTM Neural Network Incorporating Attention Mechanism for Online Ride-Hailing Demand Forecasting

Accurately predicting online ride-hailing demand can help operators allocate vehicle resources on demand, avoid idle time, and improve traffic conditions. However, due to the randomness and complexity of online ride-hailing demand data, which are affected by many factors and mostly time-series in nature, it is difficult to forecast accurately and effectively based on traditional forecasting models. Therefore, this study proposes an online ride-hailing demand forecasting model based on the attention mechanism of a random forest (RF) combined with a symmetric bidirectional long short-term memory (BiLSTM) neural network (Att-RF-BiLSTM). The model optimizes the inputs and can use past and future data to forecast, improving the forecasting precision of online ride-hailing demand. The model utilizes a random forest to filter and optimize the input variables to reduce the neural network complexity, and then an attention mechanism was incorporated into the BiLSTM neural network to construct a demand forecasting model and validate it using actual Uber pickup data from New York City. Compared with other forecasting models (Att-XGBoost-BiLSTM, Att-BiLSTM, and pure LSTM), the results show that the proposed symmetrical Att-RF-BiLSTM online ride-hailing demand forecasting model has a higher forecasting precision and fitting degree, which indicates that the proposed model can be satisfactorily applied to the area of online ride-hailing demand.

A Fault Diagnosis Technique for Wind Turbine Gearbox: An Approach using Optimized BLSTM Neural Network with Undercomplete Autoencoder

The gearbox is one of the critical components of a wind turbine. Proactive maintenance of wind turbine gearboxes is crucial to decrease maintenance and operational costs and the long downtime of the complete system. As the gearbox is a significant part of the wind turbine, a fault in the gearbox leads to the breakdown of the wind turbine system. Hence, it is important to study and analyze the faults in wind turbine gearbox systems. In this article, a neural network-based model, a Bidirectional Long Short-Term Memory (BLSTM) fused with an autoencoder is intended to categorize the condition of the gearbox into good or bad (broken tooth) condition. Feature learning and reduction are achieved extensively through the autoencoder. This improves the performance of the BLSTM model regarding time complexity and classification accuracy. This model has been applied with time series vibration data of the gearbox in a wind turbine system. The suggested model's performance is analyzed using an openly available wind turbine gearbox vibration dataset. The result showed that BLSTM accuracy with an under-complete autoencoder is highly robust and appropriate for the health monitoring of wind turbine gearbox systems using time series data. Also, in order to illustrate the advantage of the projected model for fault analysis and diagnosis in wind turbine gearbox, the throughput or time complexity of training and testing of the split dataset is compared with the conventional BLSTM model.

Breast Cancer Prediction and Control Using BiLSTM and Two-Dimensional Convolutional Neural Network

Breast cancer has a devastating effect on women. Different strategies of breast cancer classification exist with minimal work done on the prediction of the occurrence of the disease in potential carriers. In this study, a breast cancer predictive system has been developed using bidirectional long short-term memory (BiLSTM) for feature extraction and learning while the two-dimensional convolutional neural network (CNN) was used for breast cancer classification. Histopathological images were used for cancer prediction. Python was used as the programming language for implementing the system. The model was tested using datasets from The Cancer Imaging Archive (TCIA) repository. An accuracy level of 98.8% (higher than the most recent existing model) was achieved for the prediction of the future occurrence of breast cancer based on the tests on the dataset. The application of the model using live data from women can help in the prediction and control of the occurrence of breast cancer amongst women.

Short-term electricity price forecasting based on similarity day screening, two-layer decomposition technique and Bi-LSTM neural network

Electricity price forecasting (EPF) has been challenged by the widespread grid integration of renewable energy (RE), so it is critical to develop a highly accurate and reliable EPF model. In this study, novel considerations of RE generation factors are made, and a quantitative model for the impact of RE on electricity prices is built using random forests (RF) and improved Mahalanobis Distance (IMD). To reduce data duplication, similar days of EPF are first selected. Then it is suggested to decompose the electricity price series into multiple intrinsic mode functions (IMF) and residuals with different frequencies using a two-layer decomposition model based on improved comprehensive ensemble empirical mode decomposition (ICEEMD) and variational mode decomposition (VMD), in order to reduce data noise and volatility. Finally, EPF model based on Bi-directional long short-term memory (Bi-LSTM) is established to forecast multiple subsequences, and the final price forecasting result is obtained after integrated processing. Experimental results show that the RF-IMD-ICEEMD-VMD-Bi-LSTM hybrid model can significantly improve the forecasting performance, reduce the prediction error of EPF, and has the best performance among the comparison models.

Fake News Detection Using LSTM in TensorFlow and Deep Learning

Today's culture has seen a significant rise in fake news, especially with the emergence of social media platforms where misinformation may spread swiftly. False information may have detrimental effects, such as influencing elections, encouraging hate speech, and eroding trust in reliable news sources. Identification of false news has become a hot topic in recent years, with several solutions being proposed for the problem. The paper discusses LSTM (Long Short-Term Memory), Bidirectional LSTM (BiLSTM), and Convolution Neural Network (CNN)-based deep learning-based algorithms for identifying fake news. The "ISOT Fake News dataset," "News Dataset from TI-CNN," and "Getting Real About Fake News dataset" are among the datasets that were used. Following preprocessing methods such as stop keyword removal, stemming, and tokenization are used, these datasets are subjected to word embedding. This processed data is used to train the LSTM model, which determines whether or not news reports are fake. Performance metrics including precision, recall, accuracy, and F1-score provide as proof of the recommended model's efficacy in identifying fake news. Comparisons with other state-of-the-art models show its improved efficacy. In terms of both accuracy and F1-score, the CNN beat the standard LSTM and BiLSTM models. CNN-BiLSTM is most effective model, having superior findings as well as efficiency across the three datasets.

Evaluating different deep learning models for efficient extraction of Raman signals from CARS spectra.

The nonresonant background (NRB) contribution to the coherent anti-Stokes Raman scattering (CARS) signal distorts the spectral line shapes and thus degrades the chemical information. Hence, finding an effective approach for removing NRB and extracting resonant vibrational signals is a challenging task. In this work, a bidirectional LSTM (Bi-LSTM) neural network is explored for the first time to remove the NRB in the CARS spectra automatically, and the results are compared with those of three DL models reported in the literature, namely, convolutional neural network (CNN), long short-term memory (LSTM) neural network, and very deep convolutional autoencoders (VECTOR). The results of the synthetic test data have shown that the Bi-LSTM model accurately extracts the spectral lines throughout the range. In contrast, the other three models' efficiency deteriorated while predicting the peaks on either end of the spectra, which resulted in a 60 times higher mean square error than that of the Bi-LSTM model. The Pearson correlation analysis demonstrated that Bi-LSTM model performance stands out from the rest, where 94% of the test spectra have correlation coefficients of more than 0.99. Finally, these four models were evaluated on four complex experimental CARS spectra, namely, protein, yeast, DMPC, and ADP, where the Bi-LSTM model has shown superior performance, followed by CNN, VECTOR, and LSTM. This comprehensive study provides a giant leap toward simplifying the analysis of complex CARS spectroscopy and microscopy.

Lightweight Neural Network for Spatiotemporal Filling of Data Gaps in Sea Surface Temperature Images

Optical remotely sensed data often have data gaps due to cloud coverage, which hinders their full potential in many environmental applications. The question of how to accurately and effectively reconstruct the measurements in the data gaps remains a challenge. In this study, we developed a bidirectional long short-term memory (BiLSTM) model with a novel and adaptable custom temporal penalty layer for spatiotemporal gap-filling. The model was trained and tested in time-series daily night-time sea surface temperature (SST) images acquired by the Himawari-8 satellite. The modelling results showed strong performance, accurately reconstructing spatial and temporal features of cloud affected SST data. Our neural network achieved a per-image MAE of 0.1193°C and per-image RMSE of 0.1293°C. The model was also able to produce realistic SST time-series predictions that were consistent with the expected seasonal variables. Importantly, the BiLSTM model outperformed the previous state-of-the-art Simple Spatial Gapfilling Processor (SSGP) algorithm in terms of error metrics, structural similarity, and computational efficiency. Overall, the results of this study indicates that the BiLSTM neural network model we developed is highly suitable for spatiotemporal gap-filling of SST and other remotely sensed data.

1067. Predicting next 7-day Discharges of Hospitalized COVID-19 Patients Using Ensemble Learning

Abstract Background COVID-19 pandemic, especially during resurgences of cases in hard-hit areas, led to significant shortage of hospital beds. Such shortages may be alleviated through timely and effective forecasting of hospital discharges. The objective of this study is to predict next 7-day discharges of hospitalized COVID-19 patients using daily-based electronic health records (EHR) data. Methods Using EHR data of hospitalized COVID-19 patients from 03/2020-08/2021, we employed ensemble learning to predict next 7-day discharges of individual patients. We used both baseline and daily inpatient features for model training, validation, and test. Baseline features include demographic and clinical characteristics, and comorbidities. The daily inpatient features were vital signs, laboratory tests, medications administered, acute physiological scores, use of ventilator, and use of intensive care unit. 1832 hospitalized patients were identified (12,397 hospital days). Samples were randomly split at patient level (7:2:1) into training set (N=1,283 patients with 8,704 hospital days), validation set (N=366 patients with 2,524 days), and test/holdout set (patient N=183, and 1,169 days). Prediction models were trained on the training set and the validation set. We conducted the model training separately on the samples of admission day and the samples of days after admission day. The predictions were based on the ensemble learning from decision tree, XGBoost, logistic regression, and multilayer perceptron, long short-term memory (LSTM), bi-directional LSTM, and convolutional neural network. The combination of ensemble learning on the test/holdout set was used for final next 7-day predictions based on ‘hard’ voting (by majority). Where there was a tie, we used ‘soft’ voting (sum of probabilities) to break the tie. Figure 1.Data Processing PipelineFigure 2.Ensemble Model Architecture Results The overall average hospital length of stay was 8.7 (SD=10.5) days. The ensemble learning accuracies for admission-day samples and after-admission-day samples were 0.781 and 0.793, and the F1-scores for were 0.761 and 0.789, respectively. Conclusion EHR data of hospitalized COVID-19 patients can be used to predict next 7-day hospital discharges. Additional inpatient features and more advanced machine learning techniques are needed for prediction accuracy improvement. Disclosures All Authors: No reported disclosures.

Real-Time Neural Classifiers for Sensor and Actuator Faults in Three-Phase Induction Motors

The main steps involved in a fault-tolerant control (FTC) scheme are the detection of failures, isolation and reconfiguration of control. Fault detection and isolation (FDI) is a topic of interest due to its importance for the controller, since it provides the necessary information to adjust and mitigate the effects of the fault. Generally, the most common failures occur in the actuator or in sensors, so this article proposes a novel model-free scheme for the detection and isolation of sensor and actuator faults of induction motors (IM). The proposed methodology performs the task of detecting and isolating faults over data streams just after the occurrence of the failure of an induction motor (IM), by the occurrence of either disconnection, degradation, failure, or connection damage. Our approach proposes deep neural networks that do not need a nominal model or generate residuals for fault detection, which makes it a useful tool. In addition, the fault-isolation approach is carried out by classifiers that differentiate characteristics independently of the other classifiers. The long short-term memory (LSTM) neural network, bidirectional LSTM, multilayer perceptron and convolutional neural network are used for this task. The proposed sensors’ and actuator’s fault detection and isolation scheme is simple. It can be applied to various problems involving fault detection and isolation schemes. The results show that deep neural networks are a powerful and versatile tool for fault detection and isolation over data streams.

A Learning-Based Vehicle-Cloud Collaboration Approach for Joint Estimation of State-of-Energy and State-of-Health

The state-of-energy (SOE) and state-of-health (SOH) are two crucial quotas in the battery management systems, whose accurate estimation is facing challenges by electric vehicles' (EVs) complexity and changeable external environment. Although the machine learning algorithm can significantly improve the accuracy of battery estimation, it cannot be performed on the vehicle control unit as it requires a large amount of data and computing power. This paper proposes a joint SOE and SOH prediction algorithm, which combines long short-term memory (LSTM), Bi-directional LSTM (Bi-LSTM), and convolutional neural networks (CNNs) for EVs based on vehicle-cloud collaboration. Firstly, the indicator of battery performance degradation is extracted for SOH prediction according to the historical data; the Bayesian optimization approach is applied to the SOH prediction combined with Bi-LSTM. Then, the CNN-LSTM is implemented to provide direct and nonlinear mapping models for SOE. These direct mapping models avoid parameter identification and updating, which are applicable in cases with complex operating conditions. Finally, the SOH correction in SOE estimation achieves the joint estimation with different time scales. With the validation of the National Aeronautics and Space Administration battery data set, as well as the established battery platform, the error of the proposed method is kept within 3%. The proposed vehicle-cloud approach performs high-precision joint estimation of battery SOE and SOH. It can not only use the battery historical data of the cloud platform to predict the SOH but also correct the SOE according to the predicted value of the SOH. The feasibility of vehicle-cloud collaboration is promising in future battery management systems.

Intelligent Recognition Algorithm of Multiple Myocardial Infarction Based on Morphological Feature Extraction

Myocardial infarction is a type of heart disease marked by rapid progression and high mortality. In this paper, a novel intelligent recognition algorithm of multiple myocardial infarctions using a bidirectional long short-term memory (BiLSTM) neural network classification was proposed. This algorithm was based on morphological feature extraction, which can greatly improve the diagnostic efficiency of doctors for different kinds of myocardial infarction diseases. The algorithm includes noise reduction and beat segmentation of electrocardiogram (ECG) signals from the Physikalisch-Technische Bundesanstalt (PTB) database. According to the medical diagnosis guide, the distance feature of the whole waveform and the amplitude feature of the branch lead waveform are extracted. According to the extracted features, the long short-term memory network (LSTM) and the BiLSTM neural networks are built to classify and recognize heartbeats. The experimental results show that the accuracy of the morphological feature + BiLSTM algorithm in MI detection is 99.4%. At the same time, among the six common myocardial infarction diseases, the location and recognition rate of the culprit vessel is high. The sensitivity, specificity, PPV, NPV, and F1 score parameters all reach more than 98.4%, and the kappa coefficient also reaches 0.983, while the overall accuracy reaches 98.6%. The accuracy of this algorithm is improved by at least 1% compared with that of other existing algorithms. Thus, this study exhibits a very important clinical application value.

A Novel Technique for Data Quality Improvement in Human Health PHM

Human healthcare data cover different signals related to the functioning of the human body such as blood pressure, blood glucose, heart rate, among others. This kind of data plays an important role on ill patients in the intensive care unit. Unfortunately, the recorded data may include connection & human errors, measurement errors due to the movement of patients, among others. These errors are better known as Artifacts and they should be removed in case the data needs to be used for clinical purposes. Different methods have been proposed for artifact detection; however, the existing methods only analyze one signal at a time or rely on feature engineering. In this study, we present an alternative solution for artifact detection that integrates signals such as Intracranial Pressure (ICP), Electrocardiogram (ECG), and Arterial Blood Pressure (ABP). Time raw domain data, Fourier transform, and the combination of both were studied as the input of neural network models. Four deep learning algorithms were employed: convolutional neural network (CNN), long short-term memory (LSTM), bidirectional LSTM (BiLSTM), and transformer neural network. By performing a cross-validation ensemble using a dataset of 39 patients with noisy signals, the Fourier transform and CNN (FT-CNN) model outperformed the other deep learning models in terms of accuracy and computational time. This study shows that deep learning, Fourier transform, and cross-validation ensemble combined have a great potential in data quality improvement in healthcare.

Remaining Useful Life Prediction of Milling Cutters Based on CNN-BiLSTM and Attention Mechanism

Machining tools are a critical component in machine manufacturing, the life cycle of which is an asymmetrical process. Extracting and modeling the tool life variation features is very significant for accurately predicting the tool’s remaining useful life (RUL), and it is vital to ensure product reliability. In this study, based on convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM), a tool wear evolution and RUL prediction method by combining CNN-BiLSTM and attention mechanism is proposed. The powerful CNN is applied to directly process the sensor-monitored data and extract local feature information; the BiLSTM neural network is used to adaptively extract temporal features; the attention mechanism can selectively study the important degradation features and extract the tool wear status information. By evaluating the performance and generalization ability of the proposed method under different working conditions, two datasets are applied for experiments, and the proposed method outperforms the traditional method in terms of prediction accuracy.

Predicting Blood Glucose in Type-1 Diabetic Patients using Deep Learning Techniques - Approaches and Challenges

Diabetes Mellitus (DM) is a metabolic disorder where the body fails to produce the digestive hormone insulin, or the body’s ability to respond to insulin is limited. This situation leads to abnormal metabolism of carbohydrates and elevated blood sugar level. Type-1 diabetes (T1DM) is a condition arises mainly due to auto immunity disorder in which the immunity cells of the body mistakenly destroy the beta cells in the pancreas, which produce insulin. T1DM patients require to have proper control of their blood glucose level through proper medication, physical activities and continuous monitoring of blood glucose levels. A wide range of advanced wellbeing innovations, particularly computerized applications, have been growing quickly to assist individuals with dealing with their diabetes. Artificial Intelligence is a rapidly growing field, and its applications to diabetes research are becoming significantly more quickly. This paper is a review of six studies of existing neural networkbased models for the prediction of future blood glucose level in T1DM patients and describes some challenges to predict future blood glucose with the available data. These models include prediction of blood glucose level using Convolutional Neural Networks (CNN), Feedforward Neural Network (FNN), Recurrent Neural Network (RNN) implemented using Long Short Term Memory (LSTM), Convolutional Recurrent Neural Network (CRNN), Bidirectional LSTM (BiLSTM) and Dilated Recurrent Neural Networks (DRNN)

Self-propagated chaotic dynamically enhanced optical physical layer encryption communication system based on bidirectional long short-term memory neural network.

The physical layer chaotic encryption of optical communication is considered as an effective secure communication technology, which can protect data and be compatible with existing networks. Theoretically, any chaotic system or chaotic map has ideal complex dynamics. However, due to the limited precision of simulation software and digital equipment, the chaotic system often degrades dynamics, which hinders the further application of digital chaotic system in many fields. In this paper, we propose a self-propagated nonlinear chaotic dynamical enhanced optical physical layer encryption scheme based on bidirectional long short-term memory neural network (Bi-LSTM-NN). The Bi-LSTM-NN is used to train and learn the dynamical enhanced chaotic sequences with different initial values iteratively, and finally the chaotic sequences with self-propagated dynamical enhancement are output. The correlation coefficient (CC) of chaotic sequences by the enhanced chaotic system and Bi-LSTM-NN are more than 0.98. Compared with the original chaotic system, the range of sample entropy above 0.8 is more than 2 times, and the sensitivity of the initial value x0 is up to 2.28 times, and y0 is up to 1.3 times, making the key space reaches 10520. The scheme successfully encrypts constellation points and information in the frequency domain. In addition, the scheme achieves encrypted 16 quadrature amplitude modulation-orthogonal frequency division multiplexing (16QAM-OFDM) signal transmission of 65.9 Gb/s using 2 km 7-core optical fiber. The experimental results show that the scheme can ensure data security, and in the future optical network has a good application prospect.

Prediction Technology of Uncontrolled Ammunition Landing Point Based on Bi-LSTM

Landing point prediction is a commonly used algorithm in the field of guidance and control, and has high control accuracy. However, for the prediction of landing points, most algorithms do not realize end-to-end calculation, and the real-time calculation is poor. Aiming at the problem of poor real-time performance of landing calculation, a landing state prediction model based on Bi-LSTM (Bi-directional Long Short-Term Memory) neural network is proposed, and a variable step learning strategy is introduced on this basis. The model is built by taking the state of the ammunition at a certain moment (ballistic inclination, velocity, height) as the model input, and taking the landing point and landing velocity as the output of the model. And carry out simulation verification, and compare the training data fitting accuracy with the Back Propagation (BP) neural network and the LSTM neural network, as well as the complete ballistic fitting accuracy. The results show that the proposed model has a good effect in solving the problem of ammunition drop state prediction.

Predicting vacant parking space availability: A DWT-Bi-LSTM model

Accurate and efficient prediction of vacant parking space availability, despite its great significance, is no easy a task. How to address the noise in the original data and how to improve the efficiency and accuracy of prediction are among the thorny problems many existing prediction methods are confronted with. To overcome these problems, this paper proposes a DWT-Bi-LSTM model for parking space availability prediction based on historical parking data. This model combines wavelet transform (WT) and bidirectional long short-term memory (Bi-LSTM). Firstly, a multi-scale decomposition of the time series is performed using WT, and the detailed series at each scale are de-noised using the threshold method. Next, a LSTM model based on Bi-LSTM neural network is established to learn from the historical denoised data and the predicted time series are further trained to effectively avoid large prediction error. Finally, the prediction accuracy is further improved by taking advantage of the capacity of the forward and backward LSTM of capturing time-series and long-range dependence. The effectiveness of the proposed model is verified using the real-world data of a parking lot in Chongqing, China and the experimental results show that compared with other prediction methods, the proposed model demonstrates higher prediction accuracy and faster training speed under the same conditions.