Convolutional Bidirectional LSTM Research Articles

Medical Image Segmentation Using Grey Wolf-Based U-Net with Bi-Directional Convolutional LSTM

In recent years, deep learning-based networks have been able to achieve state-of-the-art performance in medical image segmentation. U-Net, one of the currently available networks, has proven to be effective when applied to the segmentation of medical images. A Convolutional Neural Network’s (CNN) performance is heavily dependent on the network’s architecture and associated parameters. There are many layers and parameters that need to be set up in order to manually create a CNN, making it a complex procedure. Designing a network is made more difficult by using a variety of connections to increase the network’s complexity. Evolutionary computation can be used to set the parameters of CNN and/or organize the CNN layers as an optimization strategy. This paper proposes an automatic evolutionary method for detecting an optimal network topology and its parameters for the segmentation of clinical image using Grey Wolf Optimization algorithm. Also, Bi-Directional LSTM integrated in the skip connection to extract dense feature characteristics of image by combining feature maps extracted from encoded and previous decoded path in nonlinear way (MIS-GW-U-Net-BiDCLSTM) is proposed. The experimental results demonstrate that the proposed method attains 98.49% accuracy with minimal parameters, which is much better than that of the other methods.

HA-ResNet: Residual Neural Network With Hidden Attention for ECG Arrhythmia Detection Using Two-Dimensional Signal.

Arrhythmia is an abnormal heart rhythm, a common clinical problem in cardiology. Long-term or severe arrhythmia may lead to stroke and sudden cardiac death. The electrocardiogram (ECG) is the most commonly used tool to diagnose arrhythmia. However, the traditional diagnosis relies on experts for manual interpretation, which is time-consuming and laborious. In recent years, many automatic arrhythmia detection methods have emerged due to advancements in deep learning. These methods can reduce manual intervention and improve diagnostic efficiency. However, extracting useful features from raw ECG signals for arrhythmia detection is still challenging due to the low frequency of ECG signals and noise distribution. In this paper, we propose a novel hidden attention residual network (HA-ResNet) for automated arrhythmia classification. In this model, the one-dimensional ECG signals are first converted into two-dimensional images and fed into an embedding layer to obtain the relevant shallow features in ECG. Then, a hidden attention layer combining Squeeze-and-Excitation (SE) block and Bidirectional Convolutional LSTM (BConvLSTM) is used to further capture the deep Spatio-temporal features. We evaluate our HA-ResNet on two public datasets and achieve F1 scores of 96.0%, 96.7%, and 87.6% on 2s segments, 5s segments, and 10s segments, respectively, which significantly outperform the existing state-of-the-art approaches. The experimental results demonstrate the effectiveness and generalization of our method.

Brain image segmentation of the corpus callosum by combining Bi-Directional Convolutional LSTM and U-Net using multi-slice CT and MRI

Background and objectiveTraditional disease diagnosis is usually performed by experienced physicians, but misdiagnosis or missed diagnosis still exists. Exploring the relationship between changes in the corpus callosum and multiple brain infarcts requires extracting corpus callosum features from brain image data, which requires addressing three key issues. (1) automation, (2) completeness, and (3) accuracy. Residual learning can facilitate network training, Bi-Directional Convolutional LSTM (BDC-LSTM) can exploit interlayer spatial dependencies, and HDC can expand the receptive domain without losing resolution. MethodsIn this paper, we propose a segmentation method by combining BDC-LSTM and U-Net to segment the corpus callosum from multiple angles of brain images based on computed tomography (CT) and magnetic resonance imaging (MRI) in which two types of sequence, namely T2-weighted imaging as well as the Fluid Attenuated Inversion Recovery (Flair), were utilized. The two-dimensional slice sequences are segmented in the cross-sectional plane, and the segmentation results are combined to obtain the final results. Encoding, BDC- LSTM, and decoding include convolutional neural networks. The coding part uses asymmetric convolutional layers of different sizes and dilated convolutions to get multi-slice information and extend the convolutional layers' perceptual field. ResultsThis paper uses BDC-LSTM between the encoding and decoding parts of the algorithm. On the image segmentation of the brain in multiple cerebral infarcts dataset, accuracy rates of 0.876, 0.881, 0.887, and 0.912 were attained for the intersection of union (IOU), dice similarity coefficient (DS), sensitivity (SE), and predictive positivity value (PPV). The experimental findings demonstrate that the algorithm outperforms its rivals in accuracy. ConclusionThis paper obtained segmentation results for three images using three models, ConvLSTM, Pyramid-LSTM, and BDC-LSTM, and compared them to verify that BDC-LSTM is the best method to perform the segmentation task for faster and more accurate detection of 3D medical images. We improve the convolutional neural network segmentation method to obtain medical images with high segmentation accuracy by solving the over-segmentation problem.

Hybrid attention-based temporal convolutional bidirectional LSTM approach for wind speed interval prediction.

Precise wind speed prediction is crucial for the management of the wind power generation systems. However, the stochastic nature of the wind speed makes optimal interval prediction very complicated. In this paper, a hybrid approach consisting of improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN), temporal convolutional network with attention mechanism (ATCN), and bidirectional long short-term memory network (Bi-LSTM) is proposed for wind speed interval prediction (WSIP). First, ICEEMDAN is used to pre-process the raw data by decomposing the wind signal to several intrinsic mode functions. ATCN is used to reduce the uncertainty from the denoised data and extract the important temporal and spatial characteristics. Then, Bi-LSTM is used to forecast the high-quality intervals for the wind speed. Existing approaches observe a decline in the forecasting performance when the time ahead increases. As a result, the hybrid approach is evaluated using 5-min, 10-min, and 30-min ahead WSIP. To evaluate the novelty of the proposed approach, an experiment is conducted utilising wind speed data from the Garden City, Manhattan wind farm. The experimental results demonstrate that the proposed framework outperformed the comparison models with percentage improvements of 36%, 47%, and 17% for 5-min, 10-min, and 30-min ahead WSIP.

Tool wear prediction based on convolutional bidirectional LSTM model with improved particle swarm optimization

Tool wear prediction based on convolutional bidirectional LSTM model with improved particle swarm optimization

Activity detection and counting people using Mask-RCNN with bidirectional ConvLSTM

Image Incorporation concerns, including background confusion, uneven population distribution, and variations in scale and familiarity, can make group counting difficult. Pre-existing information and multi-level contextual representations are required to handle these problems effectively with deep neural networks and Mask-RCNN. Numerous studies on crowd counting use density maps without segmentation, which treat a group of individuals as a single entity. This article offers a hybrid method for crowd counting that combines Mask-RCNN (MRCNN) and a bidirectional convolutional long-term memory network (ConvLSTM), dubbed (CC: MRCNN-biCLSTM). The CC: MRCNN-biCLSTM is based on the Mask-RCN; it first segments instances and generates density maps, which are passed into adversarial learning during the training phase. Finally, the bidirectional convolutional LSTM is being used to return metrics and counts for individuals within a group of individuals. Following that, the suggested activity detection technique based on the Bayesian non-linear filter AD-BNF is used to identify a person’s activity. Additionally, the suggested approach resolves human grouping and enhances metric performance. Extensive studies demonstrate that the suggested method outperforms more sophisticated techniques on four frequently used difficult criteria for density map precision and quality.

Exploratory Analysis and Feature Selection for the Prediction of Nitrogen Dioxide

Abstract. Nitrogen dioxide is one of the most hazardous pollutants identified by the World Health Organisation. Predicting and reducing pollutants is becoming a very urgent task and many methods have been used to predict their concentration, such as physical or machine learning models. In addition to choosing the right model, it is also critical to choose the appropriate features. This work focuses on the spatiotemporal prediction of nitrogen dioxide concentration using Bidirectional Convolutional LSTM integrated with the exploration of nitrogen dioxide and associated features, as well as the implementation of feature selection methods. The Root Mean Square Error and the Mean Absolute Error were used to evaluate the proposed approach.

Bidirectional convolutional LSTM for the prediction of nitrogen dioxide in the city of Madrid

Nitrogen dioxide is one of the pollutants with the most significant health effects. Advanced information on its concentration in the air can help to monitor and control further consequences more effectively, while also making it easier to apply preventive and mitigating measures. Machine learning technologies with available methods and capabilities, combined with the geospatial dimension, can perform predictive analyses with higher accuracy and, as a result, can serve as a supportive tool for productive management. One of the most advanced machine learning algorithms, Bidirectional convolutional LSTM, is being used in ongoing work to predict the concentration of nitrogen dioxide. The model has been validated to perform more accurate spatiotemporal analysis based on the integration of temporal and geospatial factors. The analysis was carried out according to two scenarios developed on the basis of selected features using data from the city of Madrid for the periods January-June 2019 and January-June 2020. Evaluation of the model’s performance was conducted using the Root Mean Square Error and the Mean Absolute Error which emphasises the superiority of the proposed model over the reference models. In addition, the significance of a feature selection technique providing improved accuracy was underlined. In terms of execution time, due to the complexity of the Bidirectional convolutional LSTM architecture, convergence and generalisation of the data took longer, resulting in the superiority of the reference models.

Multiscale Attention U-Net for Skin Lesion Segmentation

Skin cancer is the most common type of cancer in the world and it is more treatable if diagnosed early. The diagnosis process usually starts with segmenting the skin lesion area and planning a follow-up treatment by the dermatologists. Thus, the segmentation process plays a critical role in the treatment process. In recent years, machine learning methods, especially deep convolutional neural networks are proposed to address the segmentation challenge. The common segmentation methods (e.g., U-Net) deploy a series of encoding blocks to model the local representation and subsequently a series of decoding blocks to capture the semantic relation. However, these structures are usually limited to model multi-scale objects with large variations in texture and shape. To address these limitations, we propose a Multi-Scale Attention U-Net (MSAU-Net) for skin lesion segmentation. In particular, we improve the typical U-net by inserting an attention mechanism at the bottleneck of the network to model the hierarchical representation. The attention module aggregates the multi-level representation in a non-linear fashion to selectively adjust the representative features. Then it deploys a bidirectional convolutional LSTM structure to fetch the common discriminative features and suppress the less informative ones. We incorporate the resulted features in each block of the decoding path to highlight the important regions. We have evaluated our proposed network in three public skin lesion datasets, including ISIC 2017, ISIC 2018, and PH2 datasets. The experimental results demonstrate that the proposed pipeline outperforms the existing alternatives.

Predicting the Wood Mean Moisture Content in a Conventional Kiln-based Drying Process: A Data-driven Approach

The quality of the production process is the biggest concern of a company to retain their clients and be competitive on the market. In the wood production industry, the wood moisture content is one of the most important criteria to define the final quality, price, and reliability of the lumbers. After the trees have been sawn into lumbers, the latter are dried using a conventional kiln to decrease the percentage of humidity in the wood. Thus, to control the quality of the process, the moisture content should be monitored all along the drying so it can be stopped at the right moisture content. Our approach consists of using machine learning techniques to predict the mean moisture content in the kiln throughout the drying process with a lag of ten hours. Using this lag, we will be able to know exactly when to stop the drying while giving more time for the logistics preparations. The data of real time sensor's measurements, the drying conditions and some other key performance indicators were used as inputs to predict the mean moisture content in the kiln within ten hours for every five minutes. After the feature engineering, the final inputs are selected using a hybrid Forward-Backward Stepwise Selection, and then fed to a Convolutional Bidirectional LSTM recurrent neural network which has been chosen after evaluating multiple machine learning models. The final model choice is based on its theoretical performance with an R2 of 95.24% and an MAE of 3.61% on the test dataset, and several discussions with the experts of the domain to reflect the operational perspective.

Tool wear prediction using convolutional bidirectional LSTM networks

Tool wear prediction using convolutional bidirectional LSTM networks

A hybrid deep neural network based on multi-time window convolutional bidirectional LSTM for civil aircraft APU hazard identification

Safety is one of the important topics in the field of civil aviation. Auxiliary Power Unit (APU) is one of important components in aircraft, which provides electrical power and compressed air for aircraft. The hazards in APU are prone to cause economic losses and even casualties. So, actively identifying the hazards in APU before an accident occurs is necessary. In this paper, a Hybrid Deep Neural Network (HDNN) based on multi-time window convolutional neural network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) neural network is proposed for active hazard identification of APU in civil aircraft. In order to identify the risks caused by different types of failures, the proposed HDNN simultaneously integrates three CNN-BiLSTM basic models with different time window sizes in parallel by using a fully connected neural network. The CNN-BiLSTM basic model can automatically extract features representing the system state from the input data and learn the time information of irregular trends in the time series data. Nine benchmark models are compared with the proposed HDNN. The comparison results show that the proposed HDNN has the highest identification accuracy. The HDNN has the most stable identification performance for data with imbalanced samples.

Bidirectional ConvLSTMXNet for Brain Tumor Segmentation of MR Images

In recent years, deep learning based networks have achieved good performance in brain tumour segmentation of MR Image. Among the existing networks, U-Net has been successfully applied. In this paper, it is propose deep-learning based Bidirectional Convolutional LSTM XNet (BConvLSTMXNet) for segmentation of brain tumor and using GoogLeNet classify tumor & non-tumor. Evaluated on BRATS-2019 data-set and the results are obtained for classification of tumor and non-tumor with Accuracy: 0.91, Precision: 0.95, Recall: 1.00 & F1-Score: 0.92. Similarly for segmentation of brain tumor obtained Accuracy: 0.99, Specificity: 0.98, Sensitivity: 0.91, Precision: 0.91 & F1-Score: 0.88.

Domain Adaptation for Food Intake Classification With Teacher/Student Learning

Automatic dietary monitoring (ADM) stands as a challenging application in wearable healthcare technologies. In this paper, we define an ADM to perform food intake classification (FIC) over throat microphone recordings. We investigate the use of transfer learning to design an improved FIC system. Although labeled data with acoustic close-talk microphones are abundant, throat data is scarce. Therefore, we propose a new adaptation framework based on teacher/student learning. The teacher network is trained over high-quality acoustic microphone recordings, whereas the student network distills deep feature extraction capacity of the teacher over a parallel dataset. Our approach allows us to transfer the representational capacity, adds robustness to the resulting model, and improves the FIC through throat microphone recordings. The classification problem is formulated as a spectra-temporal sequence recognition using the Convolutional LSTM (ConvLSTM) models. We evaluate the proposed approach using a large scale acoustic dataset collected from online recordings, an in-house food intake throat microphone dataset, and a parallel speech dataset. The bidirectional ConvLSTM network with the proposed domain adaptation approach consistently outperforms the SVM- and CNN-based baseline methods and attains 85.2% accuracy for the classification of 10 different food intake items. This translates to 17.8% accuracy improvement with the proposed domain adaptation.

Bidirectional Convolutional LSTM Autoencoder for Risk Detection

Bidirectional Convolutional LSTM Autoencoder for Risk Detection

Estimating 3-dimensional liver motion using deep learning and 2-dimensional ultrasound images.

The main purpose of this study is to construct a system to track the tumor position during radiofrequency ablation (RFA) treatment. Existing tumor tracking systems are designed to track a tumor in a two-dimensional (2D) ultrasound (US) image. As a result, the three-dimensional (3D) motion of the organs cannot be accommodated and the ablation area may be lost. In this study, we propose a method for estimating the 3D movement of the liver as a preliminary system for tumor tracking. Additionally, in current 3D movement estimation systems, the motion of different structures during RFA could reduce the tumor visibility in US images. Therefore, we also aim to improve the estimation of the 3D movement of the liver by improving the liver segmentation. We propose a novel approach to estimate the relative 6-axial movement (x, y, z, roll, pitch, and yaw) between the liver and the US probe in order to estimate the overall movement of the liver. We used a convolutional neural network (CNN) to estimate the 3D displacement from two-dimensional US images. In addition, to improve the accuracy of the estimation, we introduced a segmentation map of the liver region as the input for the regression network. Specifically, we improved the extraction accuracy of the liver region by using a bi-directional convolutional LSTM U-Net with densely connected convolutions (BCDU-Net). By using BCDU-Net, the accuracy of the segmentation was dramatically improved, and as a result, the accuracy of the movement estimation was also improved. The mean absolute error for the out-of-plane direction was 0.0645mm/frame. The experimental results show the effectiveness of our novel method to identify the movement of the liver by BCDU-Net and CNN. Precise segmentation of the liver by BCDU-Net also contributes to enhancing the performance of the liver movement estimation.

Strong Spatiotemporal Radar Echo Nowcasting Combining 3DCNN and Bi-Directional Convolutional LSTM

In order to solve the existing problems of easy spatiotemporal information loss and low forecast accuracy in traditional radar echo nowcasting, this paper proposes an encoding-forecasting model (3DCNN-BCLSTM) combining 3DCNN and bi-directional convolutional long short-term memory. The model first constructs dimensions of input data and gets 3D tensor data with spatiotemporal features, extracts local short-term spatiotemporal features of radar echoes through 3D convolution networks, then utilizes constructed bi-directional convolutional LSTM to learn global long-term spatiotemporal feature dependencies, and finally realizes the forecast of echo image changes by forecasting network. This structure can capture the spatiotemporal correlation of radar echoes in continuous motion fully and realize more accurate forecast of moving trend of short-term radar echoes within a region. The samples of radar echo images recorded by Shenzhen and Hong Kong meteorological stations are used for experiments, the results show that the critical success index (CSI) of this proposed model for eight predicted echoes reaches 0.578 when the echo threshold is 10 dBZ, the false alarm ratio (FAR) is 20% lower than convolutional LSTM network (ConvLSTM), and the mean square error (MSE) is 16% lower than the real-time optical flow by variational method (ROVER), which outperforms the current state-of-the-art radar echo nowcasting methods.

Building Footprint Extraction from High Resolution Aerial Images Using Generative Adversarial Network (GAN) Architecture

Building extraction with high accuracy using semantic segmentation from high-resolution remotely sensed imagery has a wide range of applications like urban planning, updating of geospatial database, and disaster management. However, automatic building extraction with non-noisy segmentation map and obtaining accurate boundary information is a big challenge for most of the popular deep learning methods due to the existence of some barriers like cars, vegetation cover and shadow of trees in the high-resolution remote sensing imagery. Thus, we introduce an end-to-end convolutional neural network called Generative Adversarial Network (GAN) in this study to tackle these issues. In the generative model, we utilized SegNet model with Bi-directional Convolutional LSTM (BConvLSTM) to generate the segmentation map from Massachusetts building dataset containing high-resolution aerial imagery. BConvLSTM combines encoded features (containing of more local information) and decoded features (containing of more semantic information) to improve the performance of the model even with the presence of complex backgrounds and barriers. The adversarial training method enforces long-range spatial label vicinity to tackle with the issue of covering building objects with the existing occlusions such as trees, cars and shadows and achieve high-quality building segmentation outcomes under the complex areas. The quantitative results obtained by the proposed technique with an average F1-score of 96.81% show that the suggested approach could achieve better results through detecting and adjusting the difference between the segmentation model output and the reference map compared to other state-of-the-art approaches such as autoencoder method with 91.36%, SegNet+BConvLSTM with 95.96%, FCN-CRFs with 95.36%% SegNet with 94.77%, and GAN-SCA model with 96.36% accuracy.

Global Semantic Consistency Network for Image Manipulation Detection

This letter focuses on image manipulation detection which aims to recognize the manipulated regions under the contextual semantic information. Existing approaches usually overlook the semantic discrepancy between different levels of feature maps, and directly fuse (e.g., addition, or concatenation) them for detection. In this letter, we argue that the semantic gap is the main reason for the low effectiveness of feature fusion in manipulation predictions. To address this problem, we propose a Global Semantic Consistency Network (GSCNet) for image manipulation detection, which is based on an encoder-decoder structure. Specifically, to make GSCNet include more global texture information which has been empirically confirmed to be beneficial to manipulation detection, gram block is first deployed on each level of feature maps in the encoding stage. Based on that, bi-directional convolutional LSTM is further implemented on the decoding stage, such that feature maps of the same level have semantic consistency. Experimental results on NIST16, and CASIA v1.0 declare that GSCNet can accurately locate the manipulated regions. Furthermore, compared to the existing models, GSCNet can achieve new state-of-the-art results.

An ensemble framework based on convolutional bi-directional LSTM with multiple time windows for remaining useful life estimation

Effectively estimating remaining useful life (RUL) is crucially important for evaluating machine health. In the industry, there exists a high degree of inconsistency among the length of condition monitoring data. Thus, we propose an ensemble framework based on convolutional bi-directional long short-term memory with multiple time windows (MTW CNN-BLSTM Ensemble) for accurately predicting RUL under this circumstance. In the training phase, multiple CNN-BLSTM base models with different time window sizes are trained to capture various temporal dependencies between features. This setting expands the time window size and reduces the training error compared to traditional static time window size approaches. In the testing phase, test units are classified and suitable base models are applied according to the length of running time. A weighted average method is exploited to aggregate base models’ outcomes. This ensemble strategy can increase the utilization rate of the test data and further enhance prediction accuracy. The effectiveness of this framework is validated and the comparison with state-of-the-art methods available has been provided. The results have shown that this framework can achieve the minimum prediction error and provide stable support for equipment health management.