Dilated Convolution Module Research Articles

Automatic fabric defect detection using a wide-and-light network

Automatic fabric defect detection systems improve the quality of textile production across the industry. To make these automatic systems accessible to smaller businesses, one potential solution is to use limited memory capacity chips that can be used with hardware platforms with limited resources. That is to say, the fabric defect detection algorithm must ensure high detection accuracy while maintaining a low computational cost. Therefore, we propose a wide-and-light network structure based on Faster R-CNN for detecting common fabric defects. We enhance the feature extraction capability of the feature extraction network by designing a dilated convolution module. In a dilated convolution module, a multi-scale convolution kernel is used to adapt to defects of different sizes. Dilated convolutions can increase receptive fields without increasing the number of parameters used. Therefore, we replace a subset of ordinary convolutions with dilated convolutions to learn target features and use convolution kernel decomposition and bottleneck methods to simplify the feature extraction networks. Then, high-level semantic features are fused with bottom-level detail features (via skip-connection) to obtain multi-scale fusion features. Finally, a series of anchor frames (of different sizes) is designed to suit multi-scale fabric defect detection. Experiments show that compared with various mainstream target detection algorithms, our proposed algorithm can improve the accuracy of fabric defect detection and reduce the size of the model.

Deep Learning-Based Vein Localization on Embedded System

Venipuncture is a common process in medical treatment. In the fight against pandemic like COVID-19, it is often very difficult for medical staff to carry out venipuncture accurately, since the staff have to wear safety glasses and surgical gloves. In this work, we designed an embedded system which implements deep learning algorithm to localize veins from color skin images. The proposed method consists of a fully convolutional neural network (CNN) as encoder and feature extractor, a dilated convolution module, and a transposed convolution module as decoder. A synchronized RGB/Near Infrared (NIR) image database was constructed to provide the mapping information between the two image fields. A combined loss function which includes a per-pixel loss and a perceptual loss was presented to optimize the network parameters. To make the model adaptive to different images, a histogram specification scheme was adopted to transform the color style of an image. The model was then implemented on a NVIDIA Jetson TX2 development kit. Comprehensive experiments were conducted on different databases to evaluate the proposed method and the embedded system. Experimental results showed that the system has satisfactory performance and a promising perspective in daily medical treatment.

Integrated Multiscale Appearance Features and Motion Information Prediction Network for Anomaly Detection.

The rise of video-prediction algorithms has largely promoted the development of anomaly detection in video surveillance for smart cities and public security. However, most current methods relied on single-scale information to extract appearance (spatial) features and lacked motion (temporal) continuity between video frames. This can cause a loss of partial spatiotemporal information that has great potential to predict future frames, affecting the accuracy of abnormality detection. Thus, we propose a novel prediction network to improve the performance of anomaly detection. Due to the objects of various scales in each video, we use different receptive fields to extract detailed appearance features by the hybrid dilated convolution (HDC) module. Meanwhile, the deeper bidirectional convolutional long short-term memory (DB-ConvLSTM) module can remember the motion information between consecutive frames. Furthermore, we use RGB difference loss to replace optical flow loss as temporal constraint, which greatly reduces the time for optical flow extraction. Compared with the state-of-the-art methods in the anomaly-detection task, experiments prove that our method can more accurately detect abnormalities in various video surveillance scenes.

Clinical Target Volume Auto-Segmentation of Esophageal Cancer for Radiotherapy After Radical Surgery Based on Deep Learning.

Radiotherapy plays an important role in controlling the local recurrence of esophageal cancer after radical surgery. Segmentation of the clinical target volume is a key step in radiotherapy treatment planning, but it is time-consuming and operator-dependent. This paper introduces a deep dilated convolutional U-network to achieve fast and accurate clinical target volume auto-segmentation of esophageal cancer after radical surgery. The deep dilated convolutional U-network, which integrates the advantages of dilated convolution and the U-network, is an end-to-end architecture that enables rapid training and testing. A dilated convolution module for extracting multiscale context features containing the original information on fine texture and boundaries is integrated into the U-network architecture to avoid information loss due to down-sampling and improve the segmentation accuracy. In addition, batch normalization is added to the deep dilated convolutional U-network for fast and stable convergence. In the present study, the training and validation loss tended to be stable after 40 training epochs. This deep dilated convolutional U-network model was able to segment the clinical target volume with an overall mean Dice similarity coefficient of 86.7% and a respective 95% Hausdorff distance of 37.4 mm, indicating reasonable volume overlap of the auto-segmented and manual contours. The mean Cohen kappa coefficient was 0.863, indicating that the deep dilated convolutional U-network was robust. Comparisons with the U-network and attention U-network showed that the overall performance of the deep dilated convolutional U-network was best for the Dice similarity coefficient, 95% Hausdorff distance, and Cohen kappa coefficient. The test time for segmentation of the clinical target volume was approximately 25 seconds per patient. This deep dilated convolutional U-network could be applied in the clinical setting to save time in delineation and improve the consistency of contouring.

Optimized HRNet for image semantic segmentation

With the rapid development of deep learning, image semantic segmentation has made great progress and become a hot topic in scene understanding of computer vision. In this paper, we propose an optimized high-resolution net (HRNet) for image semantic segmentation. Unlike traditional networks usually extract feature maps based on a high-to-low encoder, which may easily loss important shape and boundary details especially for the deeper layers with lower resolutions, our optimized HRNet can maintain high resolution features at all times using a relatively shallow and parallel network structure. To improve the ability of our model in better recognizing the objects with various scales and irregular shapes, we introduce a mixed dilated convolution (MDC) module, which can not only increase the diversity of the receptive fields, but also tackle the “gridding” problem commonly existing in the conventional dilated convolution. By minimizing fine detail lost based on a DUpsample strategy, we further develop a multi-level data-dependent feature aggregation (MDFA) module to enhance the capability of our network in better identifying the fine details especially for the small objects with fuzzy boundaries. We evaluate the optimized HRNet on four different datasets, including Cityscapes, Pascal VOC2012, CamVid and the KITTI. Experimental results validate the effectiveness of our method in improving the accuracy of image semantic segmentation. Comparisons with state-of-the-art methods also verify the advantages of our optimized HRNet in achieving better semantic segmentation performance.

SA-YOLOv3: An Efficient and Accurate Object Detector Using Self-Attention Mechanism for Autonomous Driving

Object detection is becoming increasingly significant for autonomous-driving system. However, poor accuracy or low inference performance limits current object detectors in applying to autonomous driving. In this work, a fast and accurate object detector termed as SA-YOLOv3, is proposed by introducing dilated convolution and self-attention module (SAM) into the architecture of YOLOv3. Furthermore, loss function based on GIoU and focal loss is reconstructed to further optimize detection performance. With an input size of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$512\times 512$ </tex-math></inline-formula> , our proposed SA-YOLOv3 improves YOLOv3 by 2.58 mAP and 2.63 mAP on KITTI and BDD100K benchmarks, with real-time inference (more than 40 FPS). When compared with other state-of-the-art detectors, it reports better trade-off in terms of detection accuracy and speed, indicating the suitability for autonomous-driving application. To our best knowledge, it is the first method that incorporates YOLOv3 with attention mechanism, and we expect this work would guide for autonomous-driving research in the future.

Multi‐scale supervised network for crowd counting

Crowd counting is getting more and more attention in our daily life, because it can effectively prevent some safety problems. However, due to scale variations and background noise in the image, such as buildings and trees, getting the accurate number from image is a hard work. In order to address these problems, this work introduces a new multi-scale supervised network. The proposed model uses part of vgg16 model as the backbone to extract feature. In the training process, a multi-scale dilated convolution module is added at the end of each stage of the backbone network to generate attention map with different resolutions to help the model focus on the head area in feature map. In addition, the dilated convolution adopts three dilation ratios to fit different sizes of head in the image. Finally, in order to get the high-quality density map with high-resolution, the authors employ the upsampling operation to restore the density map size to the quarter size of original image. A large number of experiments on these four datasets show that the proposed network has greatly improved the counting accuracy of many existing methods.

Automated left ventricular segmentation from cardiac magnetic resonance images via adversarial learning with multi-stage pose estimation network and co-discriminator.

Left ventricular (LV) segmentation is essential for the early diagnosis of cardiovascular diseases, which has been reported as the leading cause of death all over the world. However, automated LV segmentation from cardiac magnetic resonance images (CMRI) using the traditional convolutional neural networks (CNNs) is still a challenging task due to the limited labeled CMRI data and low tolerances to irregular scales, shapes and deformations of LV. In this paper, we propose an automated LV segmentation method based on adversarial learning by integrating a multi-stage pose estimation network (MSPN) and a co-discrimination network. Different from existing CNNs, we use a MSPN with multi-scale dilated convolution (MDC) modules to enhance the ranges of receptive field for deep feature extraction. To fully utilize both labeled and unlabeled CMRI data, we propose a novel generative adversarial network (GAN) framework for LV segmentation by combining MSPN with co-discrimination networks. Specifically, the labeled CMRI are first used to initialize our segmentation network (MSPN) and co-discrimination network. Our GAN training includes two different kinds of epochs fed with both labeled and unlabeled CMRI data alternatively, which are different from the traditional CNNs only relied on the limited labeled samples to train the segmentation networks. As both ground truth and unlabeled samples are involved in guiding training, our method not only can converge faster but also obtain a better performance in LV segmentation. Our method is evaluated using MICCAI 2009 and 2017 challenge databases. Experimental results show that our method has obtained promising performance in LV segmentation, which also outperforms the state-of-the-art methods in terms of LV segmentation accuracy from the comparison results.

Automatic segmentation of bioabsorbable vascular stents in Intravascular optical coherence images using weakly supervised attention network

Coronary heart disease has become a disease with high mortality in the world. The main treatment for coronary heart disease is stent implantation, and there is now a consensus that bioabsorbable vascular stent (BVS) is the most advanced stent. However, the accuracy of current methods to detect and segment the BVS is still not effctive enough to meet the medical needs, or it is difficult to generalize. Meanwhile, due to the influence of blood artifact, the gray-based method also has great errors and uncertainties. In this paper, we propose a new framework to segment the BVS, in order to segment the contour of BVS more accurately, we use the U-Net network as the main part of the proposed network structure, add convolutional attention layer and dilated convolution module, and finally use weakly supervised learning strategy to further enhance performance. Extensive experiments demonstrate that each designed module in our proposed network can effectively improve the accuracy of the segmentation result, and when compared with other state-of-the-art methods, the overall performance on different criterias is higher.

HyperLi-Net: A hyper-light deep learning network for high-accurate and high-speed ship detection from synthetic aperture radar imagery

Ship detection from Synthetic Aperture Radar (SAR) imagery is attracting increasing attention due to its great value in ocean. However, existing most studies are frequently improving detection accuracy at the expense of detection speed. Thus, to solve this problem, this paper proposes HyperLi-Net for high-accurate and high-speed SAR ship detection. We propose five external modules to achieve high-accuracy, i.e., Multi-Receptive-Field Module (MRF-Module), Dilated Convolution Module (DC-Module), Channel and Spatial Attention Module (CSA-Module), Feature Fusion Module (FF-Module) and Feature Pyramid Module (FP-Module). We also adopt five internal mechanisms to achieve high-speed, i.e., Region-Free Model (RF-Model), Small Kernel (S-Kernel), Narrow Channel (N-Channel), Separable Convolution (Separa-Conv) and Batch Normalization Fusion (BN-Fusion). Experimental results on the SAR Ship Detection Dataset (SSDD), Gaofen-SSDD and Sentinel-SSDD show that HyperLi-Net’s accuracy and speed are both superior to the other nine state-of-the-art methods. Moreover, the satisfactory detection results on two Sentinel-1 SAR images can reveal HyperLi-Net’s good migration capability. HyperLi-Net is build from scratch with fewer parameters, lower computation costs and lighter model that can be efficiently trained on CPUs and is helpful for future hardware transplantation, e.g. FPGAs, DSPs, etc.

Stacked U-Shape Network With Channel-Wise Attention for Salient Object Detection

This paper addresses the core issue of how to learn powerful features for saliency. We have two major observations. First, feature maps of different layers in convolutional neural networks play different roles in saliency detection. Second, different feature channels in the same layer are not of equal importance to saliency, and they often have different response to foreground or background. To address these problems, a stacked U-shape network with channel-wise attention is presented to effectively utilize these features, which mainly consists of a parallel dilated convolution (PDC) module and a multi-level attention cascaded feedback (MACF) module. More specifically, PDC aims to enlarge the receptive field without increasing the computation and effectively avoid the gridding problem. MACF is innovatively designed to adaptively select the cross-layer complementary information, and the inter-dependencies between different channel maps in the same layer can be depicted well. Finally, we adopt a multi-layer loss function to improve the commonly used binary cross entropy loss which treats all pixels equally. The extensive experiments on five saliency detection datasets demonstrate that the proposed method outperforms the state-of-the-art approaches.

Learning synthetic aperture radar image despeckling without clean data

Speckle noise can reduce the image quality of synthetic aperture radar (SAR) and make interpretation more difficult. Existing SAR image despeckling convolutional neural networks require quantities of noisy–clean image pairs. However, obtaining clean SAR images is very difficult. Because continuous convolution and pooling operations result in losing many informational details while extracting the deep features of the SAR image, the quality of recovered clean images becomes worse. Therefore, we propose a despeckling network called multiscale dilated residual U-Net (MDRU-Net). The MDRU-Net can be trained directly using noisy–noisy image pairs without clean data. To protect more SAR image details, we design five multiscale dilated convolution modules that extract and fuse multiscale features. Considering that the deep and shallow features are very distinct in fusion, we design different dilation residual skip connections, which make features at the same level have the same convolution operations. Afterward, we present an effective L_hybrid loss function that can effectively improve the network stability and suppress artifacts in the predicted clean SAR image. Compared with the state-of-the-art despeckling algorithms, the proposed MDRU-Net achieves a significant improvement in several key metrics.

Pyramid Attention Dilated Network for Aircraft Detection in SAR Images

Recently, deep learning based methods have been successfully applied in synthetic aperture radar automatic target recognition (SAR ATR) fields. However, due to the effects of the special structures of aircrafts and the complexity of SAR imaging mechanism, detecting aircrafts accurately in SAR images is still challenging. To alleviate this problem, a novel network called pyramid attention dilated network (PADN) is proposed in this letter. The key component of PADN is the dilated attention block (DAB), which is composed of two submodules – multibranch dilated convolution module (MBDCM) and convolution block attention module (CBAM). In our method, MBDCM is used to enhance the relationship among discrete backscattering features of aircrafts. CBAM is employed to refine redundant information and highlight significant features of aircrafts. A well-designed fine-grained feature pyramid is established by combining the two modules reasonably into DAB when building lateral connections. To alleviate class imbalance, focal loss (FL) is employed to train our network. Experiments on a mixed SAR aircraft data set illustrate the efficiency of the proposed method for aircraft detection.

Deep Cross-Modal Image–Voice Retrieval in Remote Sensing

With the rapid progress of satellite and aircraft technologies, cross-modal remote sensing image–voice retrieval has been studied in geography recently. However, there still exist some bottlenecks: how to consider the characteristics of remote sensing data adequately and how to reduce the memory and improve the retrieval efficiency in large-scale remote sensing data. In this article, we propose a novel deep cross-modal remote sensing image–voice retrieval approach, namely, deep image–voice retrieval (DIVR), to capture more information of remote sensing data to generate hash codes with low memory and fast retrieval properties. Especially, the DIVR approach proposes inception dilated convolution module to capture multiscale contextual information of remote sensing images and voices. Moreover, in order to enhance cross-modal similarity, the deep features’ similarity term is designed to make paired similar deep features as close as possible and paired dissimilar deep features as mutually far as possible. In addition, the quantization error term is designed to drive hash-like codes to approximate hash codes, which can effectively reduce the quantization error for hash codes’ learning. Extensive experimental results on three remote sensing image–voice data sets show that the proposed DIVR approach can outperform other cross-modal retrieval approaches.

Dilated Residual Spatial Attentive Generative Adversarial Network for Single Image De-raining

This paper propose a dilated residual spatial attentive generative adversarial network for single image de-raining. The improved spatial attentive mechanism is combined with the dilated convolution residual module to optimize the Condition Generative Adversarial Network(CGAN) structure, to solve two problems still exist in the task of removing rain from a single image: First, the rain streaks contained in the dataset we can use are limited, and in the case of real rainy days, the rain streak density is diverse, it is impossible to simulate them completely and accurately. Then, the existing rain removal models cannot remove rain streaks properly for images with different rain streak density which attend to over or under rain removal. In our methods, Firstly, the dilated convolution module is used to enhance the feature extraction of rain streaks, and then an attention map is generated by the spatial attention mechanism module to accurately locate the position of rain streaks, and the rain streaks are extracted as the foreground information by combining the two parts of the network to guide the subsequent rain removal operation; Then the rain streaks is removed through the Contextual auto-encoder, combining with PatchGAN discriminator. The experimental results on two synthetic dataset and real-world rain image show that the network model proposed in this paper has high generalization ability under different rainfall conditions. it improves the rain removal effect of the model on the most realistic public dataset, and improves the recovery ability of image details while effectively removing rain on other public datasets, which verifies the effectiveness of the model in improving the generalization ability and improving the image quality after rain removal. At the same time, the effect of rain removal is better than other methods on the real rain images, which proves that the method proposed in this paper has high practical application value.

BSCN: bidirectional symmetric cascade network for retinal vessel segmentation

BackgroundRetinal blood vessel segmentation has an important guiding significance for the analysis and diagnosis of cardiovascular diseases such as hypertension and diabetes. But the traditional manual method of retinal blood vessel segmentation is not only time-consuming and laborious but also cannot guarantee the accuracy and efficiency of diagnosis. Therefore, it is especially significant to create a computer-aided method of automatic and accurate retinal vessel segmentation.MethodsIn order to extract the blood vessels’ contours of different diameters to realize fine segmentation of retinal vessels, we propose a Bidirectional Symmetric Cascade Network (BSCN) where each layer is supervised by vessel contour labels of specific diameter scale instead of using one general ground truth to train different network layers. In addition, to increase the multi-scale feature representation of retinal blood vessels, we propose the Dense Dilated Convolution Module (DDCM), which extracts retinal vessel features of different diameters by adjusting the dilation rate in the dilated convolution branches and generates two blood vessel contour prediction results by two directions respectively. All dense dilated convolution module outputs are fused to obtain the final vessel segmentation results.ResultsWe experimented the three datasets of DRIVE, STARE, HRF and CHASE_DB1, and the proposed method reaches accuracy of 0.9846/0.9872/0.9856/0.9889 and AUC of 0.9874/0.9941/0.9882/0.9874 on DRIVE, STARE, HRF and CHASE_DB1.ConclusionsThe experimental results show that compared with the state-of-art methods, the proposed method has strong robustness, it not only avoids the adverse interference of the lesion background but also detects the tiny blood vessels at the intersection accurately.

Small Object Detection with Multiple Receptive Fields

Small object detection has been a problem in deep learning convolutional neural network models. A multi-rate dilated convolution module is proposed to form a feature map to locate small objects. Benefiting from the fact that the dilated convolution does not add extra complexity while maintaining the characteristics of the high-resolution feature map, this paper replaces the traditional convolution network by setting the dilated convolution with the ratio of 1, 2, and 5, and fuses the different convolution rate convolutions. The feature map branch, extracting high-level features, using the FPN algorithm as the basic network framework, combining a variety of receptive field feature maps, enhancing the detection ability of the algorithm for small objects, and the convergence accuracy is greatly improved. Experiments show that the proposed method achieves 81.9% MAP in the PASCAL VOC2007 dataset, which exceeds the traditional object detection algorithm.

A Gated Dilated Convolution with Attention Model for Clinical Cloze-Style Reading Comprehension.

The machine comprehension research of clinical medicine has great potential value in practical application, but it has not received sufficient attention and many existing models are very time consuming for the cloze-style machine reading comprehension. In this paper, we study the cloze-style machine reading comprehension in the clinical medical field and propose a Gated Dilated Convolution with Attention (GDCA) model, which consists of a gated dilated convolution module and an attention mechanism. Our model has high parallelism and is capable of capturing long-distance dependencies. On the CliCR data set, our model surpasses the present best model on several metrics and obtains state-of-the-art result, and the training speed is 8 times faster than that of the best model.

DSPNet: A Lightweight Dilated Convolution Neural Networks for Spectral Deconvolution With Self-Paced Learning

In the fields of industry research, infrared spectrometers are widely used in diverse applications. However, the spectrum often suffers from band overlap and random noise due to the distortion caused by the point spread function, especially for aging instruments. The problem of reconstructing the clear spectrum from the degraded spectrum is called spectrum deconvolution. Traditional partial differential equation (PDE) methods rely on distribution assumptions in the reconstructed process. This restriction makes PDE methods sensitive to tackle complex instrumental broadening effect in the dispersive IR spectrometers. Also, we need to spend much time setting the parameters of PDE models manually. These problems intuitively degrade the performances of PDE methods. In this article, we propose an end-to-end neural network framework for spectral deconvolution problem. The novelty of this article lies in its strong robustness from dilated deconvolution and self-paced learning procedure to challenge the complicated degraded spectra. Actually, the deconvolution problem is tailored to a dense prediction problem in this article. Inspired by the extensive use and excellent effects of dilated convolutions in dense prediction, a lightweight dilated convolution module is given to detect the overlaps of degraded spectra. Experimental results demonstrate that the proposed solution has an outstanding performance against many other approaches. Such improvements have the potential to facilitate industrial applications and further exploration of an unknown chemical mixture. Our framework has a good performance on feature extracting and spectrum reconstruction, even in the case of low signal-to-noise ratio.

Accuracy Improvement of UNet Based on Dilated Convolution

Though previous approach for the satellite image semantic segmentation task has already achieved reliable performance across some different benchmarks, there are still some limitations for the existing methods when faced with the scenarios of the high-resolution images. In this paper, we propose a powerful dilated convolution module and successfully apply it into our improved UNet network structure. The dilated convolution module can directly expand the receptive field of the network without reducing the resolution. Experiments on DeepGlobe Road Extraction Task have shown that the proposed method in this work can achieve the mIoU score of 63.72%, which significantly outperforms the original algorithm.