Dilated Convolutional Network Research Articles

CDADNet: Context-guided dense attentional dilated network for crowd counting

Crowd counting is a conspicuous task in computer vision owing to scale variations, perspective distortions, and complex backgrounds. Existing research usually adopts the dilated convolution network to enlarge the receptive fields to solve the problem of scale variations. However, these methods easily bring background information into the large receptive fields to generate poor quality density maps. To address this problem, we propose a novel backbone called Context-guided Dense Attentional Dilated Network (CDADNet). CDADNet contains three components: an attentional module, a context-guided module and a dense attentional dilated module. The attentional module is used to provide attention maps which can remove background information, while the context-guided module is proposed to extract multi-scale contextual information. Moreover, the dense attentional dilated module aims to generate high-granularity density maps and the cascaded strategy is used to preserve information from changing scales. To verify the feasibility of our method, we compare it to the existing approaches on five crowd counting datasets (ShanghaiTech (Part_A and Part_B), WorldEXPO’10, UCSD, UCF_CC_50). The comparison results demonstrate that CDADNet is effective and robust for various scenes.

Semi-Supervised Training of Transformer and Causal Dilated Convolution Network with Applications to Speech Topic Classification

Aiming at the audio event recognition problem of speech recognition, a decision fusion method based on the Transformer and Causal Dilated Convolutional Network (TCDCN) framework is proposed. This method can adjust the model sound events for a long time and capture the time correlation, and can effectively deal with the sparsity of audio data. At the same time, our dataset comes from audio clips cropped by YouTube. In order to reliably and stably identify audio topics, we extract different features and different loss function calculation methods to find the best model solution. The experimental results from different test models show that the TCDCN model proposed in this paper achieves better recognition results than the classification using neural networks and other fusion methods.

End-to-end dilated convolution network for document image semantic segmentation

End-to-end dilated convolution network for document image semantic segmentation

An improved fully convolution network model for change detection in mining areas using sentinel-2 images

ABSTRACT High-resolution satellite images have recently been widely used for change detection of mineral resources and mining area environments. However, with their increasing resolution and number of feature sub-categories, traditional machine learning methods have become sensitive to pseudo-changes and intra-class changes, which are prone to producing misclassifications. To address these problems, a novel fully convolutional network, the Hybrid Dilated Convolutional Siamese Network (HDC-Siam) was proposed. This model combines the modular dilated convolution network with the fully convolutional Siamese networks structure, in order to reduce the commission rate. In this paper, pairs of Sentinel-2 images with an interval of about two years was used as the experimental data. The HDC-Siam model was used to detect changes, where we evaluated the accuracy in the Dongsheng coalfield in Ordos, China and the Kuznetsk coalfield in Kemerovo, Russia. We obtained F 1-scores of 85% and 75% for these respective locations. In addition, we conducted comparative experiments using two other methods – Fully Convolutional Siamese – Difference (FC-Siam-diff) and Fully Convolutional Early Fusion (FC-EF) – in order to verify that the HDC-Siam works.

A multi-objective predictive energy management strategy for residential grid-connected PV-battery hybrid systems based on machine learning technique

This paper proposes a multi-objective predictive energy management strategy based on machine learning technique for residential grid-connected hybrid energy systems. The hybrid system considered in this study comprise three principal components: a photovoltaic array as a renewable energy source, a battery bank as an energy storage system, and residential building as an electric load. The proposed strategy comprises three levels of controls: a logical level to manage the computational load and accuracy, a dual prediction model based on residual causal dilated convolutional networks for energy production and electric load on system, and a multi-objective optimization for efficient trade of energy with the utility grid by battery charge scheduling. The prediction model used in this study can provide one-step ahead photovoltaic energy production and load forecast with sufficient accuracy using a sliding window training technique and can be implemented on an average personal computer. The energy management problem comprises multiple objectives that include minimization of energy bought from utility grid, maximization the battery bank’s state-of-charge and reduction of carbon dioxide emission. The optimization problem is constrained to the maximum allowed carbon dioxide production and battery bank’s state-of-charge limits. The proposed strategy is tested for static and dynamic electricity prices using hourly energy and load data. Simulation results show a high coefficient of determination of 93.08% for energy production predictions and 97.25% for electric load predictions using proposed dual prediction model. The proposed prediction model is benchmarked against naïve prediction, support vector machine and artificial neural network models using several metrics and shows noticeable improvements in prediction accuracy. Not only the proposed strategy combined with the proposed prediction model can handle over 50% of the total yearly load requirement but also shows a significant decrease in electricity bill and carbon dioxide compared to residential buildings without hybrid energy systems and hybrid energy system without energy management strategy.

Multilevel feature fusion dilated convolutional network for semantic segmentation

Recently, convolutional neural network (CNN) has led to significant improvement in the field of computer vision, especially the improvement of the accuracy and speed of semantic segmentation tasks, which greatly improved robot scene perception. In this article, we propose a multilevel feature fusion dilated convolution network (Refine-DeepLab). By improving the space pyramid pooling structure, we propose a multiscale hybrid dilated convolution module, which captures the rich context information and effectively alleviates the contradiction between the receptive field size and the dilated convolution operation. At the same time, the high-level semantic information and low-level semantic information obtained through multi-level and multi-scale feature extraction can effectively improve the capture of global information and improve the performance of large-scale target segmentation. The encoder–decoder gradually recovers spatial information while capturing high-level semantic information, resulting in sharper object boundaries. Extensive experiments verify the effectiveness of our proposed Refine-DeepLab model, evaluate our approaches thoroughly on the PASCAL VOC 2012 data set without MS COCO data set pretraining, and achieve a state-of-art result of 81.73% mean interaction-over-union in the validate set.

Multi-path dilated convolution network for haze and glow removal in nighttime images

In this paper, we address the single-image haze removal problem in nighttime scenes. The night haze removal is a severely ill-posed problem due to the presence of various visible night light sources with varying colors and non-uniform illumination. These light sources are of different shapes and introduce a noticeable amount of glow in the night scenes. To overcome these effects, we introduce a deep learning-based DeGlow–DeHaze iterative model which accounts for varying colors and glows. The proposed model is a linear combination of three terms: the direct transmission attenuation, airlight and glow. First, a multi-path dilated convolution DeGlow network is introduced to interactively learn the local features with different reception fields and reduce the glow effect. The glow term is estimated by a binary mask that informs the location of the illumination source. As a result, the nighttime image is only left with only direct transmission and airlight terms. Finally, a separate post-processing DeHaze network is included to remove the haze effect from the reduced glow image. For our model training, we collected the night hazy images from internal and external resources, synthesized transmission maps from the NYU depth datasets, and consequently restored the haze-free images. The quantitative and qualitative evaluations show the effectiveness of our model on several real and synthetic images and compare our results with existing night haze models. The experimental results demonstrate that our multi-path CNN model outperforms other state-of-the-art methods in terms of PSNR (19.25 dB), SSIM (0.9958) evaluation parameters and computation time.

Semantic segmentation of citrus-orchard using deep neural networks and multispectral UAV-based imagery

Accurately mapping farmlands is important for precision agriculture practices. Unmanned aerial vehicles (UAV) embedded with multispectral cameras are commonly used to map plants in agricultural landscapes. However, separating plantation fields from the remaining objects in a multispectral scene is a difficult task for traditional algorithms. In this connection, deep learning methods that perform semantic segmentation could help improve the overall outcome. In this study, state-of-the-art deep learning methods to semantic segment citrus-trees in multispectral images were evaluated. For this purpose, a multispectral camera that operates at the green (530–570 nm), red (640–680 nm), red-edge (730–740 nm) and also near-infrared (770–810 nm) spectral regions was used. The performance of the following five state-of-the-art pixelwise methods were evaluated: fully convolutional network (FCN), U-Net, SegNet, dynamic dilated convolution network (DDCN) and DeepLabV3 + . The results indicated that the evaluated methods performed similarly in the proposed task, returning F1-Scores between 94.00% (FCN and U-Net) and 94.42% (DDCN). It was also determined the inference time needed per area and, although the DDCN method was slower, based on a qualitative analysis, it performed better in highly shadow-affected areas. This study demonstrated that the semantic segmentation of citrus orchards is highly achievable with deep neural networks. The state-of-the-art deep learning methods investigated here proved to be equally suitable to solve this task, providing fast solutions with inference time varying from 0.98 to 4.36 min per hectare. This approach could be incorporated into similar research, and contribute to decision-making and accurate mapping of plantation fields.

Inferring room-level use of domestic space heating from room temperature and humidity measurements using a deep, dilated convolutional network

Time series data about when heating is on and off in homes can be useful for research on building energy use and occupant behaviours, particularly data at room level and at a granularity of minutes. Direct methods which measure the temperature of radiators and other heaters can be effective at producing such data, but are expensive. Indirect methods, which infer heating on- and off-times from ambient room temperature data, can be cheaper but produce more error-prone data. Existing indirect methods have however utilised relatively simple prediction algorithms based on changes in ambient temperature between closely adjacent time points. In the method presented here we have implemented several refinements to this approach:•An Artificial Neural Network algorithm is applied to the prediction task: a deep, dilated convolutional network.•A wider range of input features is utilised to base predictions upon: ambient room temperature and humidity, and external temperature and humidity.•Predictions for each time point are based on data from a wider, 600-minute, time window.•We evaluate model performance on a dataset with 10 min granularity and achieve mean precision and recall during the heating season of >=0.74 for individual time points, and >=0.82 for full heating events, outperforming comparator methods.

PhaseDCN: A Phase-Enhanced Dual-Path Dilated Convolutional Network for Single-Channel Speech Enhancement

Recent deep neural network (DNN) based single-channel speech enhancement methods have achieved remarkable results in the time-frequency (TF) magnitude domain. To further improve the quality and intelligibility of enhanced speech, the attention to phase enhancement is also increasing. In this paper, we propose a novel dilated convolutional network (DCN) model to simultaneously enhance the magnitude and phase of noisy speech. Unlike the direct complex spectral mapping methods, we take the complex spectrum of the signal as the main target and the ideal ratio mask (IRM) as the auxiliary target in a multi-target learning framework to achieve their complementary advantages. Firstly, a feature extraction module is introduced to achieve the fusion of local and long-term features. Two different targets are learned separately, but share the common feature extraction module, which is helpful to extract more general and suitable features. During the joint learning, the intermediate estimation of the IRM target in the auxiliary path, contributing as the attention gating factors, helps to distinguish the speech or non-speech components of the complex-valued signals in the main path. To leverage more fine-grained long-term contextual information, we introduce a multi-scale dilated convolution approach for feature encoding. Moreover, the proposed model is a causal system, which can fully meet the low latency requirements of real-time speech products. Experimental results show that, compared with other advanced systems, the proposed model not only has better speech denoising performance and phase estimation accuracy, but also generalizes better in the speaker, noise, and channel mismatch cases.

Rolling Bearing Fault Diagnosis Based on One-Dimensional Dilated Convolution Network With Residual Connection

As the rolling bearing is the most important part of rotating machinery, its fault diagnosis has been a research hotspot. In order to diagnose the faults of rolling bearing under different noisy environments and different load domains, a new method named one-dimensional dilated convolution network with residual connection is proposed in this paper. The proposed method uses the one-dimensional time-domain signals of rolling bearing as input. Zigzag dilated convolution is introduced into convolution neural network, which can effectively improve the receptive field of the convolutional layer. A multi-level residual connection structure with different weight coefficients is constructed, so that the lower layer features of convolution neural network can be transferred to the upper layer, which improves the feature learning ability. Moreover, in order to enhance the useful features and weaken the useless features, we add the attention module Squeeze-and-Excitation (SE) block after each sub-residual structure. By using the rolling bearing datasets, the experimental results show that the proposed method can effectively diagnose faults of rolling bearing under different noisy environments and different load domains. Compared with other methods, the proposed method has higher accuracy.

A Method upon Deep Learning for Speech Emotion Recognition

Feature extraction and emotional classification are significant roles in speech emotion recognition. It is hard to extract and select the optimal features, researchers can not be sure what the features should be. With deep learning approaches, features could be extracted by using hierarchical abstraction layers, but it requires high computational resources and a large number of data. In this article, we choose static, differential, and acceleration coefficients of log Mel-spectrogram as inputs for the deep learning model. To avoid performance degradation, we also add a skip connection with dilated convolution network integration. All representatives are fed into a self-attention mechanism with bidirectional recurrent neural networks to learn long term global features and exploit context for each time step. Finally, we investigate contrastive center loss with softmax loss as loss function to improve the accuracy of emotion recognition. For validating robustness and effectiveness, we tested the proposed method on the Emo-DB and ERC2019 datasets. Experimental results show that the performance of the proposed method is strongly comparable with the existing state-of-the-art methods on the Emo-DB and ERC2019 with 88% and 67%, respectively. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium provided the original work is properly cited.

An Adaptive Method Based on Multiscale Dilated Convolutional Network for Binaural Speech Source Localization

Most binaural speech source localization models perform poorly in unprecedentedly noisy and reverberant situations. Here, this issue is approached by modelling a multiscale dilated convolutional neural network (CNN). The time-related crosscorrelation function (CCF) and energy-related interaural level differences (ILD) are preprocessed in separate branches of dilated convolutional network. The multiscale dilated CNN can encode discriminative representations for CCF and ILD, respectively. After encoding, the individual interaural representations are fused to map source direction. Furthermore, in order to improve the parameter adaptation, a novel semiadaptive entropy is proposed to train the network under directional constraints. Experimental results show the proposed method can adaptively locate speech sources in simulated noisy and reverberant environments.

SAR image classification method based on improved capsule network

Aiming at the problem that the convolutional neural network is difficult to recognize the posture transformation of the image, which requires a large number of label samples for training. This paper proposes an improved synthetic aperture radar image classification method based on the combination of dilated convolution and capsule network. We use dilated convolution to expand the receptive field while reducing the amount of calculation, and use the capsule network to improve the classification effect when there are fewer training samples. The experimental results show that the proposed method can still achieve high classification performance after the training sample is small and the test sample undergoes pose transformation, and at the same time, it has a low computational load.

Multi-Scale and Single-Scale Fully Convolutional Networks for Sound Event Detection

Among various Sound Event Detection (SED) systems, Recurrent Neural Networks (RNN), such as long short-term memory unit and gated recurrent unit, is used to capture temporal dependencies, but it is confined in its length of temporal dependencies, resulting in a failure to model sound events with long duration. What’s more, RNN is incapable to process datasets in parallel, leading to low efficiency and low industrial value. Given these shortcomings, we propose to use dilated convolution (and causal dilated convolution) to capture temporal dependencies, as its great ability to ensure high time resolution and obtain longer temporal dependencies under the filter size and the network depth unchanged. In addition, dilated convolution can be parallelized, so it has higher efficiency and industrial value. Based on this, we propose Single-Scale Fully Convolutional Networks (SS-FCN) composed of convolutional neural networks and dilated convolutional networks, with the former to provide frequency invariance and the later to capture temporal dependencies. With the help of dilated convolution to control the length of temporal dependencies, we observe SS-FCN modeling a single length of temporal dependencies achieves superior detection performance for finite kinds of events. For better performance, we propose Multi-Scale Fully Convolutional Networks (MS-FCN), in which the feature fusion module is introduced to capture long short-term dependencies by fusing features with different length of temporal dependencies. The proposed methods achieve competitive performance on three main datasets with higher efficiency. The results show that SED systems based on Fully Convolutional Networks have further research value and potential.

Object Detection in UAV Images via Global Density Fused Convolutional Network

Object detection in Unmanned Aerial Vehicle (UAV) images plays fundamental roles in a wide variety of applications. As UAVs are maneuverable with high speed, multiple viewpoints, and varying altitudes, objects in UAV images are distributed with great heterogeneity, varying in size, with high density, bringing great difficulty to object detection using existing algorithms. To address the above issues, we propose a novel global density fused convolutional network (GDF-Net) optimized for object detection in UAV images. We test the effectiveness and robustness of the proposed GDF-Nets on the VisDrone dataset and the UAVDT dataset. The designed GDF-Net consists of a Backbone Network, a Global Density Model (GDM), and an Object Detection Network. Specifically, GDM refines density features via the application of dilated convolutional networks, aiming to deliver larger reception fields and to generate global density fused features. Compared with base networks, the addition of GDM improves the model performance in both recall and precision. We also find that the designed GDM facilitates the detection of objects in congested scenes with high distribution density. The presented GDF-Net framework can be instantiated to not only the base networks selected in this study but also other popular object detection models.

Automatic extraction of cell nuclei using dilated convolutional network

Pathological examination has been done manually by visual inspection of hematoxylin and eosin (H&E)-stained images. However, this process is labor intensive, prone to large variations, and lacking reproducibility in the diagnosis of a tumor. We aim to develop an automatic workflow to extract different cell nuclei found in cancerous tumors portrayed in digital renderings of the H&E-stained images. For a given image, we propose a semantic pixel-wise segmentation technique using dilated convolutions. The architecture of our dilated convolutional network (DCN) is based on SegNet, a deep convolutional encoder-decoder architecture. For the encoder, all the max pooling layers in the SegNet are removed and the convolutional layers are replaced by dilated convolution layers with increased dilation factors to preserve image resolution. For the decoder, all max unpooling layers are removed and the convolutional layers are replaced by dilated convolution layers with decreased dilation factors to remove gridding artifacts. We show that dilated convolutions are superior in extracting information from textured images. We test our DCN network on both synthetic data sets and a public available data set of H&E-stained images and achieve better results than the state of the art.

Attention-Based SeriesNet: An Attention-Based Hybrid Neural Network Model for Conditional Time Series Forecasting

Traditional time series forecasting techniques can not extract good enough sequence data features, and their accuracies are limited. The deep learning structure SeriesNet is an advanced method, which adopts hybrid neural networks, including dilated causal convolutional neural network (DC-CNN) and Long-short term memory recurrent neural network (LSTM-RNN), to learn multi-range and multi-level features from multi-conditional time series with higher accuracy. However, they didn’t consider the attention mechanisms to learn temporal features. Besides, the conditioning method for CNN and RNN is not specific, and the number of parameters in each layer is tremendous. This paper proposes the conditioning method for two types of neural networks, and respectively uses the gated recurrent unit network (GRU) and the dilated depthwise separable temporal convolutional networks (DDSTCNs) instead of LSTM and DC-CNN for reducing the parameters. Furthermore, this paper presents the lightweight RNN-based hidden state attention module (HSAM) combined with the proposed CNN-based convolutional block attention module (CBAM) for time series forecasting. Experimental results show our model is superior to other models from the viewpoint of forecasting accuracy and computation efficiency.

Skeleton-Based Online Action Prediction Using Scale Selection Network.

Action prediction is to recognize the class label of an ongoing activity when only a part of it is observed. In this paper, we focus on online action prediction in streaming 3D skeleton sequences. A dilated convolutional network is introduced to model the motion dynamics in temporal dimension via a sliding window over the temporal axis. Since there are significant temporal scale variations in the observed part of the ongoing action at different time steps, a novel window scale selection method is proposed to make our network focus on the performed part of the ongoing action and try to suppress the possible incoming interference from the previous actions at each step. An activation sharing scheme is also proposed to handle the overlapping computations among the adjacent time steps, which enables our framework to run more efficiently. Moreover, to enhance the performance of our framework for action prediction with the skeletal input data, a hierarchy of dilated tree convolutions are also designed to learn the multi-level structured semantic representations over the skeleton joints at each frame. Our proposed approach is evaluated on four challenging datasets. The extensive experiments demonstrate the effectiveness of our method for skeleton-based online action prediction.

An Enhanced Multimodal Stacking Scheme for Online Pornographic Content Detection

An enhanced multimodal stacking scheme is proposed for quick and accurate online detection of harmful pornographic contents on the Internet. To accurately detect harmful contents, the implicative visual features (auditory features) are extracted using a bi-directional RNN (recurrent neural network) with VGG-16 (a multilayered dilated convolutional network) to implicitly express the signal change patterns over time within each input. Using only the implicative visual and auditory features, a video classifier and an audio classifier are trained, respectively. By using both features together, one fusion classifier is also trained. Then, these three component classifiers are stacked in the enhanced ensemble scheme to reduce the false negative errors in a serial order of the fusion classifier, video classifier, and audio classifier for a quick online detection. The proposed multimodal stacking scheme yields an improved true positive rate of 95.40% and a false negative rate of 4.60%, which are superior values to previous studies. In addition, the proposed stacking scheme can accurately detect harmful contents up to 74.58% and an average rate of 62.16% faster than the previous stacking scheme. Therefore, the proposed enhanced multimodal stacking scheme can be used to quickly and accurately filter out harmful contents in the online environments.