Convolutional Bidirectional Long Short-term Memory Research Articles

Nonlinear Impairment Compensation Using Transfer Learning-Assisted Convolutional Bidirectional Long Short-Term Memory Neural Network for Coherent Optical Communication Systems

By combining the nonlinear impairment features derived from the first-order perturbation theory, we propose a nonlinear impairment compensation (NLC) scheme based on the transfer learning-assisted convolutional bidirectional long short-term Memory (CNN-BiLSTM) neural network structure. When considering the correlation of nonlinear impairment between preceding and succeeding consecutive adjacent symbols on the current moment symbol and integrating the multidimensional feature extraction and time memory characteristics of CNN-BiLSTM, the nonlinear impairment information contained in the input feature can be fully utilized to accurately predict the nonlinear impairment showing significant compensation effect. Meanwhile, transfer learning (TL) is introduced to greatly reduce the complexity of the scheme on the basis of high compensation performance. To verify the effectiveness of the proposed scheme, we construct single-channel (SC) and 5-channel 28 GBaud polarization division multiplexing 16 quadrature amplitude modulation (PDM-16QAM)/85 GBaud PDM-64QAM simulation systems, and SC and 3-channel 28 GBaud PDM-16QAM experimental systems. The experimental results show that when compared with simple recurrent neural network (SRNN) NLC and DBP 20 steps per span (DBP20StPs), the Q-factor gain of our scheme is about 1 dB and 1.7 dB in the SC system, and about 1.1 dB and 1.5 dB in the 3-channel system at the optimal launch power, respectively. It is interesting to highlight that, by applying TL to the simulation and experimental systems, our scheme based on only 5% of the training samples can achieve compensation performance comparable to or higher quality than retraining at various launch powers.

EMDLP: Ensemble multiscale deep learning model for RNA methylation site prediction.

BackgroundRecent research recommends that epi-transcriptome regulation through post-transcriptional RNA modifications is essential for all sorts of RNA. Exact identification of RNA modification is vital for understanding their purposes and regulatory mechanisms. However, traditional experimental methods of identifying RNA modification sites are relatively complicated, time-consuming, and laborious.Machine learning approaches have been applied in the procedures of RNA sequence features extraction and classification in a computational way, which may supplement experimental approaches more efficiently. Recently, convolutional neural network (CNN) and long short-term memory (LSTM) have been demonstrated achievements in modification site prediction on account of their powerful functions in representation learning. However, CNN can learn the local response from the spatial data but cannot learn sequential correlations. And LSTM is specialized for sequential modeling and can access both the contextual representation but lacks spatial data extraction compared with CNN. There is strong motivation to construct a prediction framework using natural language processing (NLP), deep learning (DL) for these reasons.ResultsThis study presents an ensemble multiscale deep learning predictor (EMDLP) to identify RNA methylation sites in an NLP and DL way. It organically combines the dilated convolution and Bidirectional LSTM (BiLSTM), which helps to take better advantage of the local and global information for site prediction.The first step of EMDLP is to represent the RNA sequences in an NLP way. Thus, three encodings, e.g., RNA word embedding, One-hot encoding, and RGloVe, which is an improved learning method of word vector representation based on GloVe, are adopted to decipher sites from the viewpoints of the local and global information. Then, a dilated convolutional Bidirectional LSTM network (DCB) model is constructed with the dilated convolutional neural network (DCNN) followed by BiLSTM to extract potential contributing features for methylation site prediction. Finally, these three encoding methods are integrated by a soft vote to obtain better predictive performance. Experiment results on m1A and m6A reveal that the area under the receiver operating characteristic(AUROC) of EMDLP obtains respectively 95.56%, 85.24%, and outperforms the state-of-the-art models. To maximize user convenience, a user-friendly webserver for EMDLP was publicly available at http://www.labiip.net/EMDLP/index.php (http://47.104.130.81/EMDLP/index.php).ConclusionsWe developed a predictor for m1A and m6A methylation sites.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12859-022-04756-1.

Robust Deep 3D Blood Vessel Segmentation Using Structural Priors.

Deep learning has enabled significant improvements in the accuracy of 3D blood vessel segmentation. Open challenges remain in scenarios where labeled 3D segmentation maps for training are severely limited, as is often the case in practice, and in ensuring robustness to noise. Inspired by the observation that 3D vessel structures project onto 2D image slices with informative and unique edge profiles, we propose a novel deep 3D vessel segmentation network guided by edge profiles. Our network architecture comprises a shared encoder and two decoders that learn segmentation maps and edge profiles jointly. 3D context is mined in both the segmentation and edge prediction branches by employing bidirectional convolutional long-short term memory (BCLSTM) modules. 3D features from the two branches are concatenated to facilitate learning of the segmentation map. As a key contribution, we introduce new regularization terms that: a) capture the local homogeneity of 3D blood vessel volumes in the presence of biomarkers; and b) ensure performance robustness to domain-specific noise by suppressing false positive responses. Experiments on benchmark datasets with ground truth labels reveal that the proposed approach outperforms state-of-the-art techniques on standard measures such as DICE overlap and mean Intersection-over-Union. The performance gains of our method are even more pronounced when training is limited. Furthermore, the computational cost of our network inference is among the lowest compared with state-of-the-art.

GPRI2Net: A Deep-Neural-Network-Based Ground Penetrating Radar Data Inversion and Object Identification Framework for Consecutive and Long Survey Lines

Ground penetrating radar (GPR) enables infrastructure inspection using consecutive and long survey lines. However, the existing GPR data processing methods may lead to distortions or dislocations in the reconstructed shapes of the detected objects or even inconsistencies of the inverted dielectric values when performing inversion or object identification directly using GPR data obtained from the consecutive and long survey lines. To overcome these issues, this study proposed a novel deep neural network (DNN) architecture named GPRI2Net to simultaneously reconstruct the permittivity maps and categorize the object class labels from the GPR data of consecutive and long survey lines. GPRI2Net combined a convolutional neural network (CNN) based on DenseUnet and a recurrent neural network (RNN) based on bidirectional convolutional long short-term memory (Bi-ConvLSTM) to exploit the contextual information in and between the B-Scan segments extracted from the GPR data of a consecutive and long survey line. In addition, GPRI2Net performed inversion and object identification simultaneously using one network, which highly shared features between the two tasks and greatly reduced the computational complexity. Validation experiments were performed at two levels: first, using synthetic data based on the tunnel liner defects model and then using a sandbox model test in a realistic scenario. The results demonstrated that GPRI2Net can reconstruct consecutive permittivity maps and categorize the object class labels from GPR data with different dominant frequencies and survey line lengths and achieved a superior performance using the synthetic data compared to several other methods. Moreover, GPRI2Net also achieved satisfactory results using real-world GPR data.

An Automatic Deep Segmentation Network for Pixel-Level Welding Defect Detection

Accurate welding defect location is of great significance to modern manufacturing, which could be used for accurate quality evaluation and precise repairing decision-making basis of different products. Nevertheless, accurate welding defect location is still a challenging task due to some complex factors, such as complex backgrounds, low contrast, weak texture, and class imbalance issue. Recently, deep learning has got great development due to its strong feature expression ability, which has been widely applied into defect detection, but it still exists certain shortcomings on segmentation tasks with the class unbalanced issue or microdefects. To address these issues, with the encoder&#x2013;decoder network architecture, a novel welding defect location method is proposed with an attention-guided segmentation network. To reduce the contextual information loss of the deep encoder module after multiple convolution and pooling operations, a multiscale feature fusion block is proposed to embed into a U-shaped network (U-Net) to acquire more information. On the basis, combined with a bidirectional convolutional long short-term memory (BiConvLSTM) block, an improved attention block is integrated into the skip connections between the encoder path and the decoder path to capture the global, long-range contexts and emphasize target regions, contributing to locate welding defect areas and enhance the segmentation ability on microdefects. Meanwhile, to address the foreground&#x2013;background class imbalance issue, a hybrid loss function combined with binary cross-entropy (BCE) and <inline-formula> <tex-math notation="LaTeX">$D_{\text {ice}}$ </tex-math></inline-formula> loss functions is proposed to effectively utilize their unique excellent characteristics for accurate defect segmentation. Experiment results on the public <i>GDXray</i> dataset show that the proposed segmentation method could obtain a competitive segmentation performance compared with other advanced segmentation models.

Sound classification using evolving ensemble models and Particle Swarm Optimization

Automatic sound classification attracts increasing research attention owing to its vast applications, such as robot navigation, environmental sensing, musical instrument classification, medical diagnosis, and surveillance. In this research, we propose an ensemble convolutional bidirectional Long Short-Term Memory (CBiLSTM) network with optimal hyper-parameter selection for undertaking sound classification. We first transform each audio signal into a spectrogram representation using the Short-time Fourier transform (STFT). A Particle Swarm Optimization (PSO) variant is subsequently proposed to optimize the learning rate, weight decay, numbers of filters and hidden units in the convolutional and BiLSTM layers, respectively, in order to extract effective spatial–temporal characteristics from the spectrogram inputs. To tackle the issue of stagnation in optimization, the proposed algorithm incorporates local exploitation using secant and Newton–Raphson methods, promising leader generation using regular and irregular super-ellipse formulae, and three-dimensional spherical search coefficients. Moreover, it takes into account multiple fused elite signals in conjunction with numerical analysis based exploitation to balance between diversification and intensification. A variety of CBiLSTM networks with distinctive optimized settings are devised. An ensemble model is then constructed by incorporating a set of three yielded networks based on a majority voting scheme. Evaluated using several audio data sets, our ensemble CBiLSTM networks outperform those with default and optimal settings identified by other search methods, existing deep architectures and state-of-the-art related studies. In addition to sound classification tasks, the proposed PSO algorithm also outperforms a number of classical and advanced search methods for solving diverse unimodal and multimodal benchmark functions with statistical significance.

Automatic cough detection from realistic audio recordings using C-BiLSTM with boundary regression

Automatic cough detection in the patients’ realistic audio recordings is of great significance to diagnose and monitor respiratory diseases, such as COVID-19. Many detection methods have been developed so far, but they are still unable to meet the practical requirements. In this paper, we present a deep convolutional bidirectional long short-term memory (C-BiLSTM) model with boundary regression for cough detection, where cough and non-cough parts need to be classified and located. We added convolutional layers before the LSTM to enhance the cough features and preserve the temporal information of the audio data. Considering the importance of the cough event integrity for subsequent analysis, the novel model includes an embedded boundary regression on the last feature map for both higher detection accuracy and more accurate boundaries. We delicately designed, collected and labelled a realistic audio dataset containing recordings of patients with respiratory diseases, named the Corp Dataset. 168 h of recordings with 9969 coughs from 42 different patients are included. The dataset is published online on the MARI Lab website (https://mari.tongji.edu.cn/info/1012/1030.htm). The results show that the system achieves a sensitivity of 84.13%, a specificity of 99.82% and an intersection-over-union (IoU) of 0.89, which is significantly superior to other related models. With the proposed method, all the criteria on cough detection significantly increased. The open source Corp Dataset provides useful material and a benchmark for researchers investigating cough detection. We propose the state-of-the-art system with boundary regression, laying the foundation for identifying cough sounds in real-world audio data.

Novel Application of Long Short-Term Memory Network for 3D to 2D Retinal Vessel Segmentation in Adaptive Optics—Optical Coherence Tomography Volumes

Adaptive optics—optical coherence tomography (AO-OCT) is a non-invasive technique for imaging retinal vascular and structural features at cellular-level resolution. Whereas retinal blood vessel density is an important biomarker for ocular diseases, particularly glaucoma, automated blood vessel segmentation tools in AO-OCT have not yet been explored. One reason for this is that AO-OCT allows for variable input axial dimensions, which are not well accommodated by 2D-2D or 3D-3D segmentation tools. We propose a novel bidirectional long short-term memory (LSTM)-based network for 3D-2D segmentation of blood vessels within AO-OCT volumes. This technique incorporates inter-slice connectivity and allows for variable input slice numbers. We compare this proposed model to a standard 2D UNet segmentation network considering only volume projections. Furthermore, we expanded the proposed LSTM-based network with an additional UNet to evaluate how it refines network performance. We trained, validated, and tested these architectures in 177 AO-OCT volumes collected from 18 control and glaucoma subjects. The LSTM-UNet has statistically significant improvement (p &lt; 0.05) in AUC (0.88) and recall (0.80) compared to UNet alone (0.83 and 0.70, respectively). LSTM-based approaches had longer evaluation times than the UNet alone. This study shows that a bidirectional convolutional LSTM module improves standard automated vessel segmentation in AO-OCT volumes, although with higher time cost.

ViolenceNet: Dense Multi-Head Self-Attention with Bidirectional Convolutional LSTM for Detecting Violence

Introducing efficient automatic violence detection in video surveillance or audiovisual content monitoring systems would greatly facilitate the work of closed-circuit television (CCTV) operators, rating agencies or those in charge of monitoring social network content. In this paper we present a new deep learning architecture, using an adapted version of DenseNet for three dimensions, a multi-head self-attention layer and a bidirectional convolutional long short-term memory (LSTM) module, that allows encoding relevant spatio-temporal features, to determine whether a video is violent or not. Furthermore, an ablation study of the input frames, comparing dense optical flow and adjacent frames subtraction and the influence of the attention layer is carried out, showing that the combination of optical flow and the attention mechanism improves results up to 4.4%. The conducted experiments using four of the most widely used datasets for this problem, matching or exceeding in some cases the results of the state of the art, reducing the number of network parameters needed (4.5 millions), and increasing its efficiency in test accuracy (from 95.6% on the most complex dataset to 100% on the simplest one) and inference time (less than 0.3 s for the longest clips). Finally, to check if the generated model is able to generalize violence, a cross-dataset analysis is performed, which shows the complexity of this approach: using three datasets to train and testing on the remaining one the accuracy drops in the worst case to 70.08% and in the best case to 81.51%, which points to future work oriented towards anomaly detection in new datasets.

Deep learning for tracing esophageal motility function over time

Background and Objective: Esophageal high-resolution manometry (HRM) is widely performed to evaluate the representation of manometric features in patients for diagnosing normal esophageal motility and motility disorders. Clinicians commonly assess esophageal motility function using a scheme termed the Chicago classification, which is difficult, time-consuming and inefficient with large amounts of data. Methods: Deep learning is a promising approach for diagnosing disorders and has various attractive advantages. In this study, we effectively trace esophageal motility function with HRM by using a deep learning computational model, namely, EMD-DL, which leverages three-dimensional convolution (Conv3D) and bidirectional convolutional long-short-term-memory (BiConvLSTM) models. More specifically, to fully exploit wet swallowing information, we establish an efficient swallowing representation method by localizing manometric features and swallowing box regressions from HRM. Then, EMD-DL learns how to identify major motility disorders, minor motility disorders and normal motility. To the best of our knowledge, this is the first attempt to use Conv3D and BiConvLSTM to predict esophageal motility function over esophageal HRM. Results: Test experiments on HRM datasets demonstrated that the overall accuracy of the proposed EMD-DL model is 91.32% with 90.5% sensitivity and 95.87% specificity. By leveraging information across swallowing motor cycles, our model can rapidly recognize esophageal motility function better than a gastroenterologist and lays the foundation for accurately diagnosing esophageal motility disorders in real time. Conclusions: This approach opens new avenues for detecting and identifying esophageal motility function, thereby facilitating more efficient computer-aided diagnosis in clinical practice.

Planetary gearbox fault diagnosis using bidirectional-convolutional LSTM networks

Gearbox fault diagnosis is expected to significantly improve the reliability, safety and efficiency of power transmission systems. However, planetary gearbox fault diagnosis remains a challenge due to complex responses caused by multiple planetary gears. Model-based gearbox fault diagnosis techniques extract hand-crafted features from sensor data based on underlying physics and statistical analysis, which are not effective in extracting spatial and temporal features automatically. While deep learning methods such as convolutional neural network (CNN) enable automatic feature extraction from multiple sensor sources, they are not capable of extracting spatial and temporal features simultaneously without losing critical feature information. To address this issue, we introduce a novel deep neural network based on bidirectional-convolutional long short-term memory (BiConvLSTM) networks to determine the type, location, and direction of planetary gearbox faults by extracting spatial and temporal features from both vibration and rotational speed measurements automatically and simultaneously. In particular, a CNN determines spatial correlations between two measurements within one time step automatically by combining signals collected from three accelerometers and one tachometer. Long short-term memory (LSTM) networks identify temporal dependencies between two adjacent time steps. By replacing input-to-state and state-to-state operations in the LSTM cell with convolutional operations, the BiConvLSTM can learn spatial correlations and temporal dependencies without losing critical features. Experimental results have shown that the BiConvLSTM network can detect the type, location, and direction of gearbox faults with higher accuracy than conventional deep learning approaches such as CNN, LSTM, and CNN-LSTM.

A contour-aware feature-merged network for liver segmentation based on shape prior knowledge

One primary challenge in liver segmentation is the fuzzy edge contour. Recently, fully convolutional neural networks (FCNs) have been widely used in liver segmentation because of their adequate feature extraction. Nevertheless, the context among liver slices is still ignored by FCN. To address this issue, we first propose a bidirectional convolutional long short-term memory (BiConvLSTM) to explore contextual information. Meanwhile, the attention gate (AG) is utilized to fuse high-dimensional information from BiConvLSTM to remove irrelevant features. Besides, Shape-Net network is proposed to extend the liver shape pattern by latent space information, which will contribute to reduce the interference of fuzzy boundaries. Finally, the improved active contour loss function with L2 norm acts as a feature constraint. Experimental results on public benchmark datasets show that the proposed method slightly outperforms other newly published methods and achieves good performance for liver segmentation.

Study on strategy of CT image sequence segmentation for liver and tumor based on U-Net and Bi-ConvLSTM

Accurate segmentation of the liver and tumors in computed tomography (CT) images is critical for intelligent computer-aided diagnosis (CAD). The commonly used segmentation methods based on fully convolutional networks (FCN) only take a single image into consideration but do not make good use of sequence information. In this paper, two more feasible sequence segmentation strategies than 3D U-net which can utilize inter-slice and intra-slice features simultaneously at the lower hardware and time cost are studied to improve the segmentation result. U-net serves as the backbone model of segmentation and Bi-directional convolutional long short-term memory (Bi-ConvLSTM) is chosen to extract and fuse the inter-slice feature. Strategy A corrects the pre-segmented results of U-net in the fusion of sequence information as a post-processing, where Mod-1, Mod-2 and Mod-3 models are built to compare the effects of width, depth, and residual structure on the modified model of sequence segmentation. Strategy B directly integrates the fusion of sequence information into the feature extraction of U-net, and then an end-to-end model called W-net is built based on it. The experiment results show that both strategies improve the liver and tumor segmentation performance in various aspects. The results based on strategy A are closer to the ground truth with less misdiagnose region: Mod-1 achieves better accuracy on liver contour segmentation because of the largest model width; Mod-2 can obtain more accurate tumor contour since the greatest depth of feature extraction process; and Mod-3 is at the average segmentation performance. Therefore, strategy A is recommended in the application of surgery planning of tumors. Strategy B achieves better space coincidence degree and less training time cost, which is more suitable for the early screening of liver cancer.

Few-shot SAR automatic target recognition based on Conv-BiLSTM prototypical network

In recent studies, synthetic aperture radar (SAR) automatic target recognition (ATR) algorithms have achieved high recognition accuracy in the moving and stationary target acquisition and recognition (MSTAR) data set. However, these algorithms usually require hundreds or more training samples of each target type. In order to extract azimuth-insensitive features in a SAR ATR task with only a few training samples, a convolutional bidirectional long short-term memory (Conv-BiLSTM) network is designed as an embedding network to map the SAR images into a new feature space where the classification problem becomes easier. Based on the embedding network, a novel few-shot SAR ATR framework called Conv-BiLSTM Prototypical Network (CBLPN) is proposed. Experimental results on the MSTAR benchmark data set have illustrated that the proposed method performs well in SAR image classification with only a few training samples.

ResBCDU-Net: A Deep Learning Framework for Lung CT Image Segmentation.

Lung CT image segmentation is a key process in many applications such as lung cancer detection. It is considered a challenging problem due to existing similar image densities in the pulmonary structures, different types of scanners, and scanning protocols. Most of the current semi-automatic segmentation methods rely on human factors therefore it might suffer from lack of accuracy. Another shortcoming of these methods is their high false-positive rate. In recent years, several approaches, based on a deep learning framework, have been effectively applied in medical image segmentation. Among existing deep neural networks, the U-Net has provided great success in this field. In this paper, we propose a deep neural network architecture to perform an automatic lung CT image segmentation process. In the proposed method, several extensive preprocessing techniques are applied to raw CT images. Then, ground truths corresponding to these images are extracted via some morphological operations and manual reforms. Finally, all the prepared images with the corresponding ground truth are fed into a modified U-Net in which the encoder is replaced with a pre-trained ResNet-34 network (referred to as Res BCDU-Net). In the architecture, we employ BConvLSTM (Bidirectional Convolutional Long Short-term Memory)as an advanced integrator module instead of simple traditional concatenators. This is to merge the extracted feature maps of the corresponding contracting path into the previous expansion of the up-convolutional layer. Finally, a densely connected convolutional layer is utilized for the contracting path. The results of our extensive experiments on lung CT images (LIDC-IDRI database) confirm the effectiveness of the proposed method where a dice coefficient index of 97.31% is achieved.

Dynamic Modification Neural Network Model for Short-term Traffic Prediction

Short-term precise prediction of network traffic can effectively help operators reasonably allocate network resources, improve network service quality and scheduling efficiency. Our model proposed in this paper uses dynamic parameters to adjust the predicted value and refit the predicted sequence for short-term traffic prediction which is called Dynamic Modification Neural Network (DMNN) model. The architecture of our model consists of a prediction module, an adjustment module and a series re-fitting module. The prediction module learns time closeness of the traffic data before the predicted point. The adjustment module takes dynamic period characteristics of data into account and generates adjusted value by linear discrete dynamic parameters. The series re-fitting module refits predicted series from the first two modules with a new hybrid loss function. In this paper, we take different neural networks in the prediction module, such as Long Short-Term Memory (LSTM), Bi-directional Long Short-Term Memory (BiLSTM), Convolutional Long Short-Term Memory (ConvLSTM) and Convolutional Bi-directional Long Short-Term Memory (ConvBiLSTM). We evaluate the performance of the proposed model by using two real-world datasets for short-term traffic flow prediction. Moreover, Experimental results show that the proposed model with dynamic modification has much higher accuracy than other models and decreases the prediction value error and time skew of the inflection points.

Real-Time Monitoring for Hydraulic States Based on Convolutional Bidirectional LSTM with Attention Mechanism.

By monitoring a hydraulic system using artificial intelligence, we can detect anomalous data in a manufacturing workshop. In addition, by analyzing the anomalous data, we can diagnose faults and prevent failures. However, artificial intelligence, especially deep learning, needs to learn much data, and it is often difficult to get enough data at the real manufacturing site. In this paper, we apply augmentation to increase the amount of data. In addition, we propose real-time monitoring based on a deep-learning model that uses convergence of a convolutional neural network (CNN), a bidirectional long short-term memory network (BiLSTM), and an attention mechanism. CNN extracts features from input data, and BiLSTM learns feature information. The learned information is then fed to the sigmoid classifier to find out if it is normal or abnormal. Experimental results show that the proposed model works better than other deep-learning models, such as CNN or long short-term memory (LSTM).

Attention Based Bidirectional Convolutional LSTM for High-Resolution Radio Tomographic Imaging

Radio tomographic imaging (RTI) is a technique for imaging the environment by using the received signal strength (RSS) measurements from a wireless sensor network. This brief considers the data fusion for a sequence of RSS measurements, and proposes an attention based bidirectional convolutional long short-term memory (LSTM) based deep learning method to achieve the high-resolution RTI. A time sequence of tomographic images of the dynamic environment can be obtained efficiently by employing the developed RTI system. The effectiveness of the presented method is demonstrated by the simulation examples.

Distributed Deep Features Extraction Model for Air Quality Forecasting

Several studies in environmental engineering emphasize the importance of air quality forecasting for sustainable development around the world. In this paper, we studied a new approach for air quality forecasting in Busan metropolitan city. We proposed a convolutional Bi-Directional Long-Short Term Memory (Bi-LSTM) autoencoder model trained using a distributed architecture to predict the concentration of the air quality particles (PM2.5 and PM10). The proposed deep learning model can automatically learn the intrinsic correlation among the pollutants in different location. Also, the meteorological and the pollution gas information at each location are fully utilized, which is beneficial for the performance of the model. We used multiple one-dimension convolutional neural network (CNN) layers to extract the local spatial features and a stacked Bi-LSTM layer to learn the spatiotemporal correlation of air quality particles. In addition, we used a stacked deep autoencoder to encode the essential transformation patterns of the pollution gas and the meteorological data, since they are very important for providing useful information that can significantly improve the prediction of the air quality particles. Finally, in order to reduce the training time and the resource consumption, we used a distributed deep leaning approach called data parallelism, which has never been used to tackle the problem of air quality forecasting. We evaluated our approach with extensive experiments based on the data collected in Busan metropolitan city. The results reveal the superiority of our framework over ten baseline models and display how the distributed deep learning model can significantly improve the training time and even the prediction accuracy.

Video saliency prediction using enhanced spatiotemporal alignment network

Due to a variety of motions across different frames, it is highly challenging to learn an effective spatiotemporal representation for accurate video saliency prediction (VSP). To address this issue, we develop an effective spatiotemporal feature alignment network tailored to VSP, mainly including two key sub-networks: a multi-scale deformable convolutional alignment network (MDAN) and a bidirectional convolutional Long Short-Term Memory (Bi-ConvLSTM) network. The MDAN learns to align the features of the neighboring frames to the reference one in a coarse-to-fine manner, which can well handle various motions. Specifically, the MDAN owns a pyramidal feature hierarchy structure that first leverages deformable convolution (Dconv) to align the lower-resolution features across frames, and then aggregates the aligned features to align the higher-resolution features, progressively enhancing the features from top to bottom. The output of MDAN is then fed into the Bi-ConvLSTM for further enhancement, which captures the useful long-time temporal information along forward and backward timing directions to effectively guide attention orientation shift prediction under complex scene transformation. Finally, the enhanced features are decoded to generate the predicted saliency map. The proposed model is trained end-to-end without any intricate post processing. Extensive evaluations on four VSP benchmark datasets demonstrate that the proposed method achieves favorable performance against state-of-the-art methods. The source codes and all the results will be released at https://github.com/cj4L/ESAN-VSP.