Convolutional Encoder-decoder Network Research Articles

Microseismic Signal Denoising and Separation Based on Fully Convolutional Encoder–Decoder Network

Denoising methods are a highly desired component of signal processing, and they can separate the signal of interest from noise to improve the subsequent signal analyses. In this paper, an advanced denoising method based on a fully convolutional encoder–decoder neural network is proposed. The method simultaneously learns the sparse features in the time–frequency domain, and the mask-related mapping function for signal separation. The results show that the proposed method has an impressive performance on denoising microseismic signals containing various types and intensities of noise. Furthermore, the method works well even when a similar frequency band is shared between the microseismic signals and the noises. The proposed method, compared to the existing methods, significantly improves the signal–noise ratio thanks to minor changes of the microseismic signal (less distortion in the waveform). Additionally, the proposed methods preserve the shape and amplitude characteristics so that it allows better recovery of the real waveform. This method is exceedingly useful for the automatic processing of the microseismic signal. Further, it has excellent potential to be extended to the study of exploration seismology and earthquakes.

Adaptive Thermal Sensor Array Placement for Human Segmentation and Occupancy Estimation

Thermal sensor array (TSA) offers privacy-preserving, low-cost, and non-invasive features, which makes it suitable for various indoor applications such as anomaly detection, health monitoring, home security, and monitoring energy efficiency applications. Previous approaches to human-centred applications using the TSA usually relied on the use of a fixed sensor location to make the human-sensor distance and the human presence shape fixed. However, placing this sensor in different locations and new indoor environments can pose a significant challenge. In this paper, a novel framework based on a deep convolutional encoder-decoder network is proposed to address this challenge in real-life deployment. The framework presents a semantic segmentation of the human presence and estimates the occupancy in indoor-environment. It is also capable to segment the human presence and counts the number of people from different sensor locations, indoor environments, and human to sensor distance. Furthermore, the impact of the distance on the human presence using the TSA is investigated. The framework is evaluated to estimate the occupancy in different sensor locations, the number of occupants, environments, and human distance with classification and regression machine learning approaches. This paper shows that the classification approach using the adaptive boosting algorithm is an accurate approach which has achieves an accuracy of 98.43% and 100% from vertical and overhead sensor locations respectively.

Multi-Channel Fetal ECG Denoising With Deep Convolutional Neural Networks.

Non-invasive fetal electrocardiography represents a valuable alternative continuous fetal monitoring method that has recently received considerable attention in assessing fetal health. However, the non-invasive fetal electrocardiogram (ECG) is typically severely contaminated by a considerable amount of various noise sources, rendering fetal ECG denoising a very challenging task. This work employs a deep learning approach for removing the residual noise from multi-channel fetal ECG after the maternal ECG has been suppressed. We propose a deep convolutional encoder-decoder network with symmetric skip-layer connections, learning end-to-end mappings from noise-corrupted fetal ECG signals to clean ones. Experiments on simulated data show an average signal-to-noise ratio (SNR) improvement of 9.5 dB for fetal ECG signals with input SNR ranging between −20 and 20 dB. The method is additionally evaluated on a large set of real signals, demonstrating that it can provide significant quality improvement of the noisy fetal ECG signals. We further show that employment of multi-channel signal information by the network provides superior and more reliable performance as opposed to its single-channel network counterpart. The presented method is able to preserve beat-to-beat morphological variations and does not require any prior information on the power spectra of the noise or the pulse location.

Pedestrian Trajectory Prediction in Heterogeneous Traffic Using Pose Keypoints-Based Convolutional Encoder-Decoder Network

Future pedestrian trajectory prediction offers great prospects for many practical applications. Most existing methods focus on social interaction among pedestrians but ignore the factors that heterogeneous traffic objects (cars, dogs, bicycles, motorcycles, etc.) have significant influence on the future trajectory of a subject pedestrian. Also, the walking direction intention of a pedestrian may be referred by his/her pose keypoints. Considering this, this work proposes to predict a pedestrian's future trajectory by jointly using neighboring heterogeneous traffic information and his/her pose keypoints. To fulfill this, an end-to-end pose keypoints-based convolutional encoder-decoder network (PK-CEN) is designed, in which the heterogeneous traffic and pose keypoints are modeled as input. After training, PK-CEN is evaluated on manifold crowded video sequences collected from the public dataset MOT16, MOT17 and MOT20. Experimental results demonstrate that it outperforms state-of-the-art approaches, in terms of prediction errors.

Context prior-based with residual learning for face detection: A deep convolutional encoder–decoder network

In the field of security, faces are usually blurry, occluded, diverse pose and small in the image captured by an outdoor surveillance camera, which is affected by the external environment such as the camera pose and range, weather conditions, etc. It can be described as a problem of hard face detection in natural images. To solve this problem, we propose a deep convolutional neural network named feature hierarchy encoder–decoder network (FHEDN). It is motivated by two observations from contextual semantic information and the mechanism of multi-scale face detection. The proposed network is a scale-variant style architecture and single stage, which are composed of encoder and decoder subnetworks. Based on the assumption that contextual semantic information around face being auxiliary to detect faces, we introduce a residual mechanism to fuse context prior-based information into face feature and formulate the learning chain to train each encoder–decoder pair. In addition, we discuss some important factors in implement details such as the distribution of training dataset, the scale of feature hierarchy, and anchor box size, etc. They have some impact on the detection performance of the final network. Compared with some state-of-the-art algorithms, our method achieves promising performance on the popular benchmarks including AFW, PASCAL FACE, FDDB, and WIDER FACE. Consequently, the proposed approach can be efficiently implemented and routinely applied to detect faces with severe occlusion and arbitrary pose variations in unconstrained scenes. Our code and results are available on https://github.com/zzxcoder/EvaluationFHEDN.

Building extraction from VHR remote sensing imagery by combining an improved deep convolutional encoder-decoder architecture and historical land use vector map

ABSTRACT Building extraction has attracted considerable attention in the field of remote sensing image analysis. Fully convolutional network modelling is a recently developed technique that is capable of significantly enhancing building extraction accuracy. It is a prominent branch of deep learning and uses advanced state-of-the-art techniques, especially with regard to building segmentation. In this paper, we present an enhanced deep convolutional encoder-decoder (DCED) network by incorporating historical land use vector maps (HVMs) customized for building extraction. The approach combines enhanced DCED architecture with multi-scale image pyramid for pixel-wise building segmentation. The improved DCED network, together with symmetrical dense-shortcut connection structures, is employed to establish the encoders for automatic extraction of building features. The feature maps from early layers were fused with more discriminative feature maps from the deeper layers through ‘Res path’ skip connections for superior building extraction accuracy. To further reduce the occurrence of falsely segmented buildings, and to sharpen the buildings’ boundaries, the new temporal testing image is segmented under the constraints of an HVM. A majority voting strategy is employed to ensure the homogeneity of the building objects as the post-processing method. Experimental results indicate that the proposed approach exhibits competitive quantitative and qualitative performance, effectively alleviating the salt-and-pepper phenomenon and block effects, and retaining the edge structures of buildings. Compared with other state-of-the-art methods, our method demonstrably achieves the optimal final accuracies.

Learning to complete partial observations from unpaired prior knowledge

We present a novel training strategy that allows convolutional encoder-decoder networks, to complete partially observed data by means of hallucination. As input, it takes data from a partially observed domain, for which no complete ground truth is available, and data from an unpaired prior knowledge domain and trains the network in an end-to-end manner. This strategy is demonstrated for the task of completing 2-D road layouts as well as 3-D vehicle shapes. In contrast to alternative approaches, our strategy is compatible with networks that use skip connections, to improve detail in the completed output, while not requiring adversarial supervision. To demonstrate its benefits, our training strategy is benchmarked against two state-of-the-art baselines, one using a two-step auto-encoder training strategy and one using an adversarial strategy. Our novel strategy achieves an improvement up to +12% F-measure on the Cityscapes dataset. The learned network intrinsically generalizes better than the baselines on unseen datasets, which is demonstrated by an improvement up to +24% F-measure on the unseen KITTI dataset. Moreover, our approach outperforms the baselines using the same backbone network on the 3-D shape completion benchmark by reducing the Hamming distance with 15%.

Image reflection removal using end‐to‐end convolutional neural network

Single image reflection removal is an ill-posed problem. To solve this problem, this study develops a network structure based on a deep encoder-decoder RRnet. Unlike most deep learning strategies applied in this context, the authors find that redundant information increases the difficulty of predicting images on the network; thus, the proposed method uses mixed reflection image cascaded edges as input to the network. The proposed network structure is divided into two parts: the first part is a deep convolutional encoder-decoder network. Its function uses the mixed reflection image and the target edge as input to predict the target layer. The second part is an identical encoder-decoder network structure. Its function uses the mixed reflection image and the reflection edge as input to predict the image reflection layer. In addition, the authors use joint loss to optimise the network model. To train the neural network, they also create an image dataset for reflection removal, which includes a true mixed reflection image and a synthetic mixed reflection image. They use four evaluation indicators to evaluate the proposed method and the other six methods. The experimental results indicate that the proposed method is superior to previous methods.

Tea Sprouts Segmentation via Improved Deep Convolutional Encoder-Decoder Network

Tea Sprouts Segmentation via Improved Deep Convolutional Encoder-Decoder Network

Brain MRI Super-Resolution Using 3D Dilated Convolutional Encoder–Decoder Network

The spatial resolution of magnetic resonance images (MRI) is limited by the hardware capacity, sampling time, signal-to-noise ratio (SNR), and patient comfort. Recently, deep convolutional neural networks (CNN) have achieved impressive success in MRI super-resolution (SR) reconstruction. Increasing network depth or width can enlarge the receptive field to improve SR accuracy, however, it is impractical for MRI reconstruction in clinical applications because of high computational loads. To address this issue, we propose a novel dilated convolutional encoder-decoder (DCED) network to improve the resolution of MRI. We exploit three-dimensional (3D) dilated convolutions as encoders to extract high-frequency features. The dilated encoders capture wider contextual information by exponentially enlarging the receptive field, without introducing additional parameters or layers. Then we decode the features using deconvolution operations to alleviate gridding artifacts and restore fine details. To improve information flow, the encoders and decoders are aggregated into symmetrically connected blocks. The output of each block is passed to the final convolution layer, which facilitates to extract hierarchical features. In addition, we also exploit a geometric self-ensemble 3D wavelet fusion method to improve the potential performance of MRI SR. Experimental results on four public available brain datasets show that our proposed method outperforms NLM (non-local means), LRTV (low-rank and total variation) and current CNN-based SR methods, which demonstrates that our method achieves a new state-of-the-art performance in MRI SR task.

Deep Neural Network-Based System for Autonomous Navigation in Paddy Field

This paper presents a novel vision based approach for detecting rows of crop in paddy field. The precise detection of crop row enables a farm-tractor to autonomously navigate the field for successful inter-row weeding. While prior works on crop row detection rely primarily on various image based features, a deep neural network based approach for learning semantic graphics to directly extract the crop rows from an input image is used in this work. A deep convolutional encoder decoder network is trained to detect the crop lines using semantic graphics. The detected crop lines are then used to derive control signal for steering the tractor autonomously in the field. The results demonstrate that the proposed method is able to detect the rows of paddy accurately and enable the tractor to navigate autonomously along the crop rows even with a simple proportional only controller.

Automatic Crack Detection and Measurement of Concrete Structure Using Convolutional Encoder-Decoder Network

The detection and measurement of crack at pixel level is a challenge to existing methods. To overcome this challenge, this paper proposes a convolutional encoder-decoder network (CedNet) to detect crack from image, and the maximum widths and orientations of cracks are measured using image post-processing techniques. To realize this, a database including 1800 crack images (with 761×569 pixel resolution) taken from concrete structures is built. Then the CedNet is designed, trained and validated using the built database. The validating results show 98.90% accuracy, 93.58% precision, 94.73% recall, 93.18% F-measure, 87.23% intersection over union (IoU) of crack and 98.82% IoU of background. Subsequently, the robustness and adaptability of the trained model is tested. To measure true maximum widths and orientations of cracks, a laboratory experiment is carried out to calibrate a relation between ratio (pixel distance / real distance) and field of view (camera's view range on concrete surface included in image) and distance from the smartphone to concrete surface. In the post-processing techniques, the perspective transformation is employed to correct distorted images caused by the existence of the oblique angles between the smartphone and concrete surfaces. Then the maximum widths and orientations of cracks in predicted results are measured respectively using the Euclidean distance transformation and least squares principle. As comparison, two existing deep learning-based crack detection and measurement method are used to examine the performance of the proposed approach. The comparison results show that the proposed method substantiates quite good performance to detect cracks and measure maximum widths and orientations of cracks in our database.

Combining Multi-Perspective Attention Mechanism With Convolutional Networks for Monaural Speech Enhancement

The redundant convolutional encoder-decoder network has been proven useful in speech enhancement tasks. This network can capture the localized time-frequency details of speech signals through the fully convolutional network structure and the feature selection capability that results from the encoder-decoder mechanism. However, extracting informative features, which we regard as important for the representational capability of speech enhancement models, is not considered explicitly. To solve this problem, we introduce the attention mechanism into the convolutional encoder-decoder model to explicitly emphasize useful information from three aspects, namely, channel, space, and concurrent space-and-channel. Furthermore, the attention operation is specifically achieved through the squeeze-and-excitation mechanism and its variants. The model can adaptively emphasize valuable information and suppress useless ones by assigning weights from different perspectives according to global information, thereby improving its representational capability. Experimental results show that the proposed attention mechanisms can employ a small fraction of parameters to effectively improve the performance of CNN-based models compared with their normal versions, and generalize well to unseen noises, signal-to-noise ratios (SNR) and speakers. Among these mechanisms, the concurrent space-channel-wise attention exhibits the most significant improvement. And when comparing with the state-of-the-art, they can produce comparable or better results. We also integrate the proposed attention mechanisms with other convolutional neural network (CNN)-based models and gain performance. Moreover, we visualize the enhancement results to show the effect of the attention mechanisms more clearly.

Automatic Building Extraction From High-Resolution Aerial Imagery via Fully Convolutional Encoder-Decoder Network With Non-Local Block

Extracting buildings automatically from high-resolution aerial images is a significant and fundamental task for various practical applications, such as land-use statistics and urban planning. Recently, various methods based on deep learning, especially the fully convolution networks, achieve impressive scores in this challenging semantic segmentation task. However, the lack of global contextual information and the careless upsampling method limit the further improvement of the performance for building extraction task. To simultaneously address these problems, we propose a novel network named Efficient Non-local Residual U-shape Network(ENRU-Net), which is composed of a well designed U-shape encoder-decoder structure and an improved non-local block named asymmetric pyramid non-local block (APNB). The encoder-decoder structure is adopted to extract and restore the feature maps carefully, and APNB could capture global contextual information by utilizing self-attention mechanism. We evaluate the proposed ENRU-Net and compare it with other state-of-the-art models on two widely-used public aerial building imagery datasets: the Massachusetts Buildings Dataset and the WHU Aerial Imagery Dataset. The experiments show that the accuracy of ENRU-Net on these datasets has remarkable improvement against previous state-of-the-art semantic segmentation models, including FCN-8s, U-Net, SegNet and Deeplab v3. The subsequent analysis also indicates that our ENRU-Net has advantages in efficiency for building extraction from high-resolution aerial images.

Learning Semantic Graphics Using Convolutional Encoder-Decoder Network for Autonomous Weeding in Paddy.

Weeds in agricultural farms are aggressive growers which compete for nutrition and other resources with the crop and reduce production. The increasing use of chemicals to control them has inadvertent consequences to the human health and the environment. In this work, a novel neural network training method combining semantic graphics for data annotation and an advanced encoder–decoder network for (a) automatic crop line detection and (b) weed (wild millet) detection in paddy fields is proposed. The detected crop lines act as a guiding line for an autonomous weeding robot for inter-row weeding, whereas the detection of weeds enables autonomous intra-row weeding. The proposed data annotation method, semantic graphics, is intuitive, and the desired targets can be annotated easily with minimal labor. Also, the proposed “extended skip network” is an improved deep convolutional encoder–decoder neural network for efficient learning of semantic graphics. Quantitative evaluations of the proposed method demonstrated an increment of 6.29% and 6.14% in mean intersection over union (mIoU), over the baseline network on the task of paddy line detection and wild millet detection, respectively. The proposed method also leads to a 3.56% increment in mIoU and a significantly higher recall compared to a popular bounding box-based object detection approach on the task of wild–millet detection.

Fetal electrocardiography extraction with residual convolutional encoder-decoder networks.

In the context of fetal monitoring, non-invasive fetal electrocardiography is an alternative approach to the traditional Doppler ultrasound technique. However, separating the fetal electrocardiography (FECG) component from the abdominal electrocardiography (AECG) remains a challenging task. This is mainly due to the interference from maternal electrocardiography, which has larger amplitude and overlaps with the FECG in both temporal and frequency domains. The main objective is to present a novel approach to FECG extraction by using a deep learning strategy from single-channel AECG recording. A residual convolutional encoder-decoder network (RCED-Net) is developed for this task of FECG extraction. The single-channel AECG recording is the input to the RCED-Net. And the RCED-Net extracts the feature of AECG and directly outputs the estimate of FECG component in the AECG recording. The AECG recordings from two different databases are collected to illustrate the efficiency of the proposed method. And the achieved results show that the proposed technique exhibits the best performance when compared to the existing methods in the literature. This work is a proof of concept that the proposed method could effectively extract the FECG component from AECG recordings. The focus on single-channel FECG extraction technique contributes to the commercial applications for long-term fetal monitoring.

Cascaded deep convolutional encoder-decoder neural networks for efficient liver tumor segmentation

Liver and hepatic tumor segmentation remains a challenging problem in Computer Tomography (CT) images analysis due to its shape variation and vague boundary. The general hypothesis says that deep learning methods produce improved results on medical image segmentation. This paper formulates the segmentation of liver tumor in CT abdominal images as a classification problem, and then solves it using a cascaded classifier framework based on deep convolutional neural networks. Two deep encoder-decoder convolutional neural networks (EDCNN) were constructed and trained to cascade segments of both the liver and lesions in CT images with limited image quantity. In other words, an EDCNN segments the liver image as the input for the training of a second EDCNN. The second EDCNN then segments the tumor regions within the liver ROI regions as predicted by the first EDCNN. Segmenting the hepatic tumor inside the liver ROI also significantly reduces false-positives. The proposed model was then tested using a public dataset (3DIRCADb), and several metrics were used in order to quantitatively evaluate its performance. The proposed method produced an average DICE score of 95.22% for the test set of CT images. The proposed method was then compared with some of the existing methods. The experimental results demonstrated that the proposed EDCNN achieved improved performance in segmentation accuracy over some existing methods.

A Skip Attention Mechanism for Monaural Singing Voice Separation

This work proposes a simple but effective attention mechanism, namely Skip Attention (SA), for monaural singing voice separation (MSVS). First, the SA, embedded in the convolutional encoder-decoder network (CEDN), realizes an attention-driven and dependency modeling for the repetitive structures of the music source. Second, the SA, replacing the popular skip connection in the CEDN, effectively controls the flow of the low-level (vocal and musical) features to the output and improves the feature sensitivity and accuracy for MSVS. Finally, we implement the proposed SA on the Stacked Hourglass Network (SHN), namely Skip Attention SHN (SA-SHN). Quantitative and qualitative evaluation results have shown that the proposed SA-SHN achieves significant performance improvement on the MIR-1K dataset (compared to the state-of-the-art SHN) and competitive MSVS performance on the DSD100 dataset (compared to the state-of-the-art DenseNet), even without using any data augmentation methods.

Vision-Based Autonomous Crack Detection of Concrete Structures Using a Fully Convolutional Encoder-Decoder Network.

The visual inspection of massive civil infrastructure is a common trend for maintaining its reliability and structural health. However, this procedure, which uses human inspectors, requires long inspection times and relies on the subjective and empirical knowledge of the inspectors. To address these limitations, a machine vision-based autonomous crack detection method is proposed using a deep convolutional neural network (DCNN) technique. It consists of a fully convolutional neural network (FCN) with an encoder and decoder framework for semantic segmentation, which performs pixel-wise classification to accurately detect cracks. The main idea is to capture the global context of a scene and determine whether cracks are in the image while also providing a reduced and essential picture of the crack locations. The visual geometry group network (VGGNet), a variant of the DCCN, is employed as a backbone in the proposed FCN for end-to-end training. The efficacy of the proposed FCN method is tested on a publicly available benchmark dataset of concrete crack images. The experimental results indicate that the proposed method is highly effective for concrete crack classification, obtaining scores of approximately 92% for both the recall and F1 average.

SEMANTIC SEGMENTATION OF AIRBORNE IMAGES AND CORRESPONDING DIGITAL SURFACE MODELS – ADDITIONAL INPUT DATA OR ADDITIONAL TASK?

Abstract. We analyze the effects of additional height data for semantic segmentation of aerial images with a convolutional encoder-decoder network. Besides a merely image-based semantic segmentation, we trained the same network with height as additional input and furthermore, we defined a multi-task model, where we trained the network to estimate the relative height of objects in parallel to semantic segmentation on the image data only. Our findings are, that excellent results are possible for image data only and additional height information has no significant effect – neither when employed as extra input nor when used for multi-task training, even with differently weighted losses. Based on our results, we, thus, hypothesize that a strong encoder-decoder network implicitly learns the correlation of object categories and relative heights.