Bidirectional Long Short-term Memory Neural Network Research Articles

Breast Cancer Prediction and Control Using BiLSTM and Two-Dimensional Convolutional Neural Network

Breast cancer has a devastating effect on women. Different strategies of breast cancer classification exist with minimal work done on the prediction of the occurrence of the disease in potential carriers. In this study, a breast cancer predictive system has been developed using bidirectional long short-term memory (BiLSTM) for feature extraction and learning while the two-dimensional convolutional neural network (CNN) was used for breast cancer classification. Histopathological images were used for cancer prediction. Python was used as the programming language for implementing the system. The model was tested using datasets from The Cancer Imaging Archive (TCIA) repository. An accuracy level of 98.8% (higher than the most recent existing model) was achieved for the prediction of the future occurrence of breast cancer based on the tests on the dataset. The application of the model using live data from women can help in the prediction and control of the occurrence of breast cancer amongst women.

Short-term electricity price forecasting based on similarity day screening, two-layer decomposition technique and Bi-LSTM neural network

Electricity price forecasting (EPF) has been challenged by the widespread grid integration of renewable energy (RE), so it is critical to develop a highly accurate and reliable EPF model. In this study, novel considerations of RE generation factors are made, and a quantitative model for the impact of RE on electricity prices is built using random forests (RF) and improved Mahalanobis Distance (IMD). To reduce data duplication, similar days of EPF are first selected. Then it is suggested to decompose the electricity price series into multiple intrinsic mode functions (IMF) and residuals with different frequencies using a two-layer decomposition model based on improved comprehensive ensemble empirical mode decomposition (ICEEMD) and variational mode decomposition (VMD), in order to reduce data noise and volatility. Finally, EPF model based on Bi-directional long short-term memory (Bi-LSTM) is established to forecast multiple subsequences, and the final price forecasting result is obtained after integrated processing. Experimental results show that the RF-IMD-ICEEMD-VMD-Bi-LSTM hybrid model can significantly improve the forecasting performance, reduce the prediction error of EPF, and has the best performance among the comparison models.

Fake News Detection Using LSTM in TensorFlow and Deep Learning

Today's culture has seen a significant rise in fake news, especially with the emergence of social media platforms where misinformation may spread swiftly. False information may have detrimental effects, such as influencing elections, encouraging hate speech, and eroding trust in reliable news sources. Identification of false news has become a hot topic in recent years, with several solutions being proposed for the problem. The paper discusses LSTM (Long Short-Term Memory), Bidirectional LSTM (BiLSTM), and Convolution Neural Network (CNN)-based deep learning-based algorithms for identifying fake news. The "ISOT Fake News dataset," "News Dataset from TI-CNN," and "Getting Real About Fake News dataset" are among the datasets that were used. Following preprocessing methods such as stop keyword removal, stemming, and tokenization are used, these datasets are subjected to word embedding. This processed data is used to train the LSTM model, which determines whether or not news reports are fake. Performance metrics including precision, recall, accuracy, and F1-score provide as proof of the recommended model's efficacy in identifying fake news. Comparisons with other state-of-the-art models show its improved efficacy. In terms of both accuracy and F1-score, the CNN beat the standard LSTM and BiLSTM models. CNN-BiLSTM is most effective model, having superior findings as well as efficiency across the three datasets.

Evaluating different deep learning models for efficient extraction of Raman signals from CARS spectra.

The nonresonant background (NRB) contribution to the coherent anti-Stokes Raman scattering (CARS) signal distorts the spectral line shapes and thus degrades the chemical information. Hence, finding an effective approach for removing NRB and extracting resonant vibrational signals is a challenging task. In this work, a bidirectional LSTM (Bi-LSTM) neural network is explored for the first time to remove the NRB in the CARS spectra automatically, and the results are compared with those of three DL models reported in the literature, namely, convolutional neural network (CNN), long short-term memory (LSTM) neural network, and very deep convolutional autoencoders (VECTOR). The results of the synthetic test data have shown that the Bi-LSTM model accurately extracts the spectral lines throughout the range. In contrast, the other three models' efficiency deteriorated while predicting the peaks on either end of the spectra, which resulted in a 60 times higher mean square error than that of the Bi-LSTM model. The Pearson correlation analysis demonstrated that Bi-LSTM model performance stands out from the rest, where 94% of the test spectra have correlation coefficients of more than 0.99. Finally, these four models were evaluated on four complex experimental CARS spectra, namely, protein, yeast, DMPC, and ADP, where the Bi-LSTM model has shown superior performance, followed by CNN, VECTOR, and LSTM. This comprehensive study provides a giant leap toward simplifying the analysis of complex CARS spectroscopy and microscopy.

Lightweight Neural Network for Spatiotemporal Filling of Data Gaps in Sea Surface Temperature Images

Optical remotely sensed data often have data gaps due to cloud coverage, which hinders their full potential in many environmental applications. The question of how to accurately and effectively reconstruct the measurements in the data gaps remains a challenge. In this study, we developed a bidirectional long short-term memory (BiLSTM) model with a novel and adaptable custom temporal penalty layer for spatiotemporal gap-filling. The model was trained and tested in time-series daily night-time sea surface temperature (SST) images acquired by the Himawari-8 satellite. The modelling results showed strong performance, accurately reconstructing spatial and temporal features of cloud affected SST data. Our neural network achieved a per-image MAE of 0.1193°C and per-image RMSE of 0.1293°C. The model was also able to produce realistic SST time-series predictions that were consistent with the expected seasonal variables. Importantly, the BiLSTM model outperformed the previous state-of-the-art Simple Spatial Gapfilling Processor (SSGP) algorithm in terms of error metrics, structural similarity, and computational efficiency. Overall, the results of this study indicates that the BiLSTM neural network model we developed is highly suitable for spatiotemporal gap-filling of SST and other remotely sensed data.

1067. Predicting next 7-day Discharges of Hospitalized COVID-19 Patients Using Ensemble Learning

Abstract Background COVID-19 pandemic, especially during resurgences of cases in hard-hit areas, led to significant shortage of hospital beds. Such shortages may be alleviated through timely and effective forecasting of hospital discharges. The objective of this study is to predict next 7-day discharges of hospitalized COVID-19 patients using daily-based electronic health records (EHR) data. Methods Using EHR data of hospitalized COVID-19 patients from 03/2020-08/2021, we employed ensemble learning to predict next 7-day discharges of individual patients. We used both baseline and daily inpatient features for model training, validation, and test. Baseline features include demographic and clinical characteristics, and comorbidities. The daily inpatient features were vital signs, laboratory tests, medications administered, acute physiological scores, use of ventilator, and use of intensive care unit. 1832 hospitalized patients were identified (12,397 hospital days). Samples were randomly split at patient level (7:2:1) into training set (N=1,283 patients with 8,704 hospital days), validation set (N=366 patients with 2,524 days), and test/holdout set (patient N=183, and 1,169 days). Prediction models were trained on the training set and the validation set. We conducted the model training separately on the samples of admission day and the samples of days after admission day. The predictions were based on the ensemble learning from decision tree, XGBoost, logistic regression, and multilayer perceptron, long short-term memory (LSTM), bi-directional LSTM, and convolutional neural network. The combination of ensemble learning on the test/holdout set was used for final next 7-day predictions based on ‘hard’ voting (by majority). Where there was a tie, we used ‘soft’ voting (sum of probabilities) to break the tie. Figure 1.Data Processing PipelineFigure 2.Ensemble Model Architecture Results The overall average hospital length of stay was 8.7 (SD=10.5) days. The ensemble learning accuracies for admission-day samples and after-admission-day samples were 0.781 and 0.793, and the F1-scores for were 0.761 and 0.789, respectively. Conclusion EHR data of hospitalized COVID-19 patients can be used to predict next 7-day hospital discharges. Additional inpatient features and more advanced machine learning techniques are needed for prediction accuracy improvement. Disclosures All Authors: No reported disclosures.

Real-Time Neural Classifiers for Sensor and Actuator Faults in Three-Phase Induction Motors

The main steps involved in a fault-tolerant control (FTC) scheme are the detection of failures, isolation and reconfiguration of control. Fault detection and isolation (FDI) is a topic of interest due to its importance for the controller, since it provides the necessary information to adjust and mitigate the effects of the fault. Generally, the most common failures occur in the actuator or in sensors, so this article proposes a novel model-free scheme for the detection and isolation of sensor and actuator faults of induction motors (IM). The proposed methodology performs the task of detecting and isolating faults over data streams just after the occurrence of the failure of an induction motor (IM), by the occurrence of either disconnection, degradation, failure, or connection damage. Our approach proposes deep neural networks that do not need a nominal model or generate residuals for fault detection, which makes it a useful tool. In addition, the fault-isolation approach is carried out by classifiers that differentiate characteristics independently of the other classifiers. The long short-term memory (LSTM) neural network, bidirectional LSTM, multilayer perceptron and convolutional neural network are used for this task. The proposed sensors’ and actuator’s fault detection and isolation scheme is simple. It can be applied to various problems involving fault detection and isolation schemes. The results show that deep neural networks are a powerful and versatile tool for fault detection and isolation over data streams.

A Learning-Based Vehicle-Cloud Collaboration Approach for Joint Estimation of State-of-Energy and State-of-Health

The state-of-energy (SOE) and state-of-health (SOH) are two crucial quotas in the battery management systems, whose accurate estimation is facing challenges by electric vehicles' (EVs) complexity and changeable external environment. Although the machine learning algorithm can significantly improve the accuracy of battery estimation, it cannot be performed on the vehicle control unit as it requires a large amount of data and computing power. This paper proposes a joint SOE and SOH prediction algorithm, which combines long short-term memory (LSTM), Bi-directional LSTM (Bi-LSTM), and convolutional neural networks (CNNs) for EVs based on vehicle-cloud collaboration. Firstly, the indicator of battery performance degradation is extracted for SOH prediction according to the historical data; the Bayesian optimization approach is applied to the SOH prediction combined with Bi-LSTM. Then, the CNN-LSTM is implemented to provide direct and nonlinear mapping models for SOE. These direct mapping models avoid parameter identification and updating, which are applicable in cases with complex operating conditions. Finally, the SOH correction in SOE estimation achieves the joint estimation with different time scales. With the validation of the National Aeronautics and Space Administration battery data set, as well as the established battery platform, the error of the proposed method is kept within 3%. The proposed vehicle-cloud approach performs high-precision joint estimation of battery SOE and SOH. It can not only use the battery historical data of the cloud platform to predict the SOH but also correct the SOE according to the predicted value of the SOH. The feasibility of vehicle-cloud collaboration is promising in future battery management systems.

Intelligent Recognition Algorithm of Multiple Myocardial Infarction Based on Morphological Feature Extraction

Myocardial infarction is a type of heart disease marked by rapid progression and high mortality. In this paper, a novel intelligent recognition algorithm of multiple myocardial infarctions using a bidirectional long short-term memory (BiLSTM) neural network classification was proposed. This algorithm was based on morphological feature extraction, which can greatly improve the diagnostic efficiency of doctors for different kinds of myocardial infarction diseases. The algorithm includes noise reduction and beat segmentation of electrocardiogram (ECG) signals from the Physikalisch-Technische Bundesanstalt (PTB) database. According to the medical diagnosis guide, the distance feature of the whole waveform and the amplitude feature of the branch lead waveform are extracted. According to the extracted features, the long short-term memory network (LSTM) and the BiLSTM neural networks are built to classify and recognize heartbeats. The experimental results show that the accuracy of the morphological feature + BiLSTM algorithm in MI detection is 99.4%. At the same time, among the six common myocardial infarction diseases, the location and recognition rate of the culprit vessel is high. The sensitivity, specificity, PPV, NPV, and F1 score parameters all reach more than 98.4%, and the kappa coefficient also reaches 0.983, while the overall accuracy reaches 98.6%. The accuracy of this algorithm is improved by at least 1% compared with that of other existing algorithms. Thus, this study exhibits a very important clinical application value.

A Novel Technique for Data Quality Improvement in Human Health PHM

Human healthcare data cover different signals related to the functioning of the human body such as blood pressure, blood glucose, heart rate, among others. This kind of data plays an important role on ill patients in the intensive care unit. Unfortunately, the recorded data may include connection & human errors, measurement errors due to the movement of patients, among others. These errors are better known as Artifacts and they should be removed in case the data needs to be used for clinical purposes. Different methods have been proposed for artifact detection; however, the existing methods only analyze one signal at a time or rely on feature engineering. In this study, we present an alternative solution for artifact detection that integrates signals such as Intracranial Pressure (ICP), Electrocardiogram (ECG), and Arterial Blood Pressure (ABP). Time raw domain data, Fourier transform, and the combination of both were studied as the input of neural network models. Four deep learning algorithms were employed: convolutional neural network (CNN), long short-term memory (LSTM), bidirectional LSTM (BiLSTM), and transformer neural network. By performing a cross-validation ensemble using a dataset of 39 patients with noisy signals, the Fourier transform and CNN (FT-CNN) model outperformed the other deep learning models in terms of accuracy and computational time. This study shows that deep learning, Fourier transform, and cross-validation ensemble combined have a great potential in data quality improvement in healthcare.

Remaining Useful Life Prediction of Milling Cutters Based on CNN-BiLSTM and Attention Mechanism

Machining tools are a critical component in machine manufacturing, the life cycle of which is an asymmetrical process. Extracting and modeling the tool life variation features is very significant for accurately predicting the tool’s remaining useful life (RUL), and it is vital to ensure product reliability. In this study, based on convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM), a tool wear evolution and RUL prediction method by combining CNN-BiLSTM and attention mechanism is proposed. The powerful CNN is applied to directly process the sensor-monitored data and extract local feature information; the BiLSTM neural network is used to adaptively extract temporal features; the attention mechanism can selectively study the important degradation features and extract the tool wear status information. By evaluating the performance and generalization ability of the proposed method under different working conditions, two datasets are applied for experiments, and the proposed method outperforms the traditional method in terms of prediction accuracy.

Predicting Blood Glucose in Type-1 Diabetic Patients using Deep Learning Techniques - Approaches and Challenges

Diabetes Mellitus (DM) is a metabolic disorder where the body fails to produce the digestive hormone insulin, or the body’s ability to respond to insulin is limited. This situation leads to abnormal metabolism of carbohydrates and elevated blood sugar level. Type-1 diabetes (T1DM) is a condition arises mainly due to auto immunity disorder in which the immunity cells of the body mistakenly destroy the beta cells in the pancreas, which produce insulin. T1DM patients require to have proper control of their blood glucose level through proper medication, physical activities and continuous monitoring of blood glucose levels. A wide range of advanced wellbeing innovations, particularly computerized applications, have been growing quickly to assist individuals with dealing with their diabetes. Artificial Intelligence is a rapidly growing field, and its applications to diabetes research are becoming significantly more quickly. This paper is a review of six studies of existing neural networkbased models for the prediction of future blood glucose level in T1DM patients and describes some challenges to predict future blood glucose with the available data. These models include prediction of blood glucose level using Convolutional Neural Networks (CNN), Feedforward Neural Network (FNN), Recurrent Neural Network (RNN) implemented using Long Short Term Memory (LSTM), Convolutional Recurrent Neural Network (CRNN), Bidirectional LSTM (BiLSTM) and Dilated Recurrent Neural Networks (DRNN)

Self-propagated chaotic dynamically enhanced optical physical layer encryption communication system based on bidirectional long short-term memory neural network.

The physical layer chaotic encryption of optical communication is considered as an effective secure communication technology, which can protect data and be compatible with existing networks. Theoretically, any chaotic system or chaotic map has ideal complex dynamics. However, due to the limited precision of simulation software and digital equipment, the chaotic system often degrades dynamics, which hinders the further application of digital chaotic system in many fields. In this paper, we propose a self-propagated nonlinear chaotic dynamical enhanced optical physical layer encryption scheme based on bidirectional long short-term memory neural network (Bi-LSTM-NN). The Bi-LSTM-NN is used to train and learn the dynamical enhanced chaotic sequences with different initial values iteratively, and finally the chaotic sequences with self-propagated dynamical enhancement are output. The correlation coefficient (CC) of chaotic sequences by the enhanced chaotic system and Bi-LSTM-NN are more than 0.98. Compared with the original chaotic system, the range of sample entropy above 0.8 is more than 2 times, and the sensitivity of the initial value x0 is up to 2.28 times, and y0 is up to 1.3 times, making the key space reaches 10520. The scheme successfully encrypts constellation points and information in the frequency domain. In addition, the scheme achieves encrypted 16 quadrature amplitude modulation-orthogonal frequency division multiplexing (16QAM-OFDM) signal transmission of 65.9 Gb/s using 2 km 7-core optical fiber. The experimental results show that the scheme can ensure data security, and in the future optical network has a good application prospect.

Prediction Technology of Uncontrolled Ammunition Landing Point Based on Bi-LSTM

Landing point prediction is a commonly used algorithm in the field of guidance and control, and has high control accuracy. However, for the prediction of landing points, most algorithms do not realize end-to-end calculation, and the real-time calculation is poor. Aiming at the problem of poor real-time performance of landing calculation, a landing state prediction model based on Bi-LSTM (Bi-directional Long Short-Term Memory) neural network is proposed, and a variable step learning strategy is introduced on this basis. The model is built by taking the state of the ammunition at a certain moment (ballistic inclination, velocity, height) as the model input, and taking the landing point and landing velocity as the output of the model. And carry out simulation verification, and compare the training data fitting accuracy with the Back Propagation (BP) neural network and the LSTM neural network, as well as the complete ballistic fitting accuracy. The results show that the proposed model has a good effect in solving the problem of ammunition drop state prediction.

Predicting vacant parking space availability: A DWT-Bi-LSTM model

Accurate and efficient prediction of vacant parking space availability, despite its great significance, is no easy a task. How to address the noise in the original data and how to improve the efficiency and accuracy of prediction are among the thorny problems many existing prediction methods are confronted with. To overcome these problems, this paper proposes a DWT-Bi-LSTM model for parking space availability prediction based on historical parking data. This model combines wavelet transform (WT) and bidirectional long short-term memory (Bi-LSTM). Firstly, a multi-scale decomposition of the time series is performed using WT, and the detailed series at each scale are de-noised using the threshold method. Next, a LSTM model based on Bi-LSTM neural network is established to learn from the historical denoised data and the predicted time series are further trained to effectively avoid large prediction error. Finally, the prediction accuracy is further improved by taking advantage of the capacity of the forward and backward LSTM of capturing time-series and long-range dependence. The effectiveness of the proposed model is verified using the real-world data of a parking lot in Chongqing, China and the experimental results show that compared with other prediction methods, the proposed model demonstrates higher prediction accuracy and faster training speed under the same conditions.

Research on GNSS interference recognition based on ROI of correlation peaks

SummaryThe general Global Navigation Satellite System (GNSS) receiver faces several challenges because of jamming signals, spoofing signals, and multipath signals, which severely influence its safety. In this paper, a receiver scheme with an interference recognition function is designed. In the latter, the correlation peak with different shapes is produced according to different interferences. The machine learning method is then applied to recognize and classify these feature maps. This transforms the interference recognition problem into a machine learning‐based classification problem. In order to reduce the complexity of the machine learning network, only the finite‐length correlation peak region of interest (ROI) is extracted as network input, endowing the shallow neural network with the interference recognition function. Afterward, five data acquisition environments are designed: authentic, spoofing, jamming, non‐line‐of‐sight (NLOS) multipath, and line‐of‐sight (LOS) multipath. Moreover, several experimental data are acquired, followed by the production of the correlation peak maps dataset, that are then learned and tested using two machine learning networks: one‐dimensional convolutional neural network (1D‐CNN) and bidirectional long short‐term memory neural network (BiLSTM‐NN). The results demonstrate that a recognition accuracy rate of over 98% can be reached using the shallow machine learning network.

A novel hybrid water quality forecast model based on real-time data decomposition and error correction

Accurate forecast for water quality is of great importance because it can support water resource management with the future information. In this research, we propose a novel hybrid model by using data decomposition, error correction, and machine learning. In our method, first, the initial forecast is obtained by a prediction model that uses improved complete ensemble empirical mode decomposition with adaptive noise and bidirectional long short-term memory (BLSTM) neural network. Next, a novel error correction framework, which is built by variational mode decomposition and BLSTM neural network, is used to improve forecast accuracy by correcting the initial forecast error. Water quality data of Poyang Lake, China is used to evaluate our model. Results indicate that our model shows highly accurate forecast performance for all of the 9 water quality datasets (the average of mean absolute percentage error (MAPE) of 7 day-ahead forecast is 2.12%; 30 day-ahead forecast is 4.06%). In addition, our model outperforms the competitor models, particularly, compared to the prediction model without error correction framework, the average of MAPE is reduced by 33.33% for 7 day-ahead forecast; 30.48% for 30 day-ahead forecast. This research demonstrates that the proposed error correction framework is an effective tool to improve forecast accuracy for water quality.

Wind Power Forecasting Method Based on Bidirectional Long Short-Term Memory Neural Network and Error Correction

With the improvement of penetration rate of wind power in the power system, its volatility and intermittence bring new problems to the power grid. The accurate forecasting of wind power is an effective way to alleviate the impact on the power grid. In this paper, a novel wind power forecasting method is proposed. Firstly, a wind power forecasting model based on the bidirectional long short-term memory (BiLSTM) neural network is established to forecast the wind power according to the wind speed of numerical weather forecasting (NWF). Secondly, a wind power forecasting error time series model based on empirical mode decomposition (EMD) is established to decrease the forecasting error. Finally, this paper uses the real data to simulate and verify the proposed method. Evaluating by the root mean square error (RMSE), symmetric mean absolute percentage error (SMAPE), and Theil inequality coefficient (TIC), the simulation results show that the forecasting accuracy of the BiLSTM neural network model are 10.25%, 6.71% and 12.18% higher than LSTM model respectively. After correcting wind power forecasting error using the proposed time series model based on EMD, the accuracy of wind power forecasting is further improved.

A wind power forecasting method based on optimized decomposition prediction and error correction

To reduce the effect of nonlinearity and volatility in the wind power time sequence, a two-stage short-term wind power forecasting method based on optimized decomposition prediction and error correction is proposed. In the first stage, in order to improve the decomposition effect of variational mode decomposition (VMD), the decomposition loss is defined as the evaluation criterion to guide the parameter setting of VMD, and flower pollination algorithm (FPA) is utilized to automatically optimize the parameters of VMD. Then the complex wind power sequence is decomposed into simple intrinsic mode functions (IMFs). Besides, bi-directional long short-term memory (BiLSTM) neural network is built for each IMF to explore the deep time-series features of wind power in both past and future directions. In the second stage, to reduce the correlation among meteorological factors, principal component analysis (PCA) is employed to convert the multi-dimensional meteorological factors into low-dimensional principal components. Then, with the input of IMFs and principal component, an error correction model based on BiLSTM neural network is established to reduce the inherent error of the model. The experimental results show that the proposed method has higher prediction accuracy than the traditional methods in single-step and multi-step ahead forecasting.

Reflection Coefficients Inversion Based on the Bidirectional Long Short-Term Memory Network

Improving the vertical resolution of seismic data to satisfy the demands for detailed characterization of reservoirs is an important part in seismic data processing. The sparse spike inversion(SSI) technique greatly enhances the seismic resolution by assuming that the reflection coefficients follow the sparse distributions. However, its characterization for the spatial structure of thin and thin interbed reservoirs is still limited. In this letter, we propose a novel approach for enhancing seismic resolution using a bidirectional long short-term memory(BiLSTM) neural network by extracting reflection coefficients directly from seismic data. By designing the network architectures, multiple seismic samples map to the specific reflection coefficient. Compared with the SSI method, the BiLSTM neural network provides higher resolution inversion results on model data, which demonstrates the effectiveness of the proposed approach for thin structure characterizations. The convergence speed of the proposed method is fast and the training process is stable. In addition, we conduct the high-resolution seismic data processing on the field data based on the transfer learning technology. High-resolution processing results illustrate the generalization ability and adaptability of the BiLSTM neural network.