Deep Convolutional Encoder-decoder Network Research Articles

Deep convolutional encoder–decoder networks based on ensemble learning for semantic segmentation of high-resolution aerial imagery

Deep convolutional encoder–decoder networks based on ensemble learning for semantic segmentation of high-resolution aerial imagery

Mild Traumatic Brain Injury andCareer Stage Associate with Visible Perivascular Spaces in Special Operations Forces Soldiers.

Mild traumatic brain injury (mTBI) and occupational blast exposure in military Service Members may lead to impaired brain waste clearance which increases neurological disease risk. Perivascular spaces (PVS) are a key part of the glymphatic system which supports brain waste clearance, preferentially during sleep. Visible PVS on clinical magnetic resonance imaging have been previously observed in patients with neurodegenerative diseases and animal neurotrauma models. The purpose of this study was to determine associations between PVS morphological characteristics, military career stage, and mTBI history in Special Operations Forces (SOF)Soldiers. Participants underwent T2-weighed neuroimaging to capture three-dimensional whole brain volumes. Segmentation was performed using a previously validated, multi-scale deep convolutional encoder-decoder neural network. Only PVS clusters within the white matter mask were quantified for analyses. Due to non-normal PVS metric distribution, non-parametric Mann-Whitney U tests were used to determine group differences in PVS outcomes. In total, 223 healthy SOF combat Soldiers (age = 33.1 ± 4.3yrs) were included, 217 reported career stage. Soldiers with mTBI history had greater PVS number (z = 2.51, P = 0.013) and PVS volume (z = 2.42, P = 0.016). In-career SOF combat Soldiers had greater PVS number (z = 2.56, P = 0.01) and PVS volume (z = 2.28, P = 0.02) compared to a baseline cohort. Mild TBI history is associated with increased PVS burden in SOF combat Soldiers that are clinically recovered from mTBI. This may indicate ongoing physiological changes that could lead to impaired waste clearance via the glymphatic system. Future studies should determine if PVS number and volume are meaningful neurobiological outcomes for neurodegenerative disease risk and if clinical interventions such as improving sleep can reduce PVS burden.

A deep learning based surrogate model for reservoir dynamic performance prediction

Predicting the reservoir dynamic performance is an important basis for exploiting the remaining oil and improving the recovery. Traditional prediction techniques primarily include numerical simulation and reservoir engineering methods, which show technical drawbacks such as low computational efficiency and too simplified assumptions. To tackle this challenge, we propose a surrogate model for reservoir dynamic performance prediction based on a Variety of View deep convolutional encoder-decoder (VoV-DCED) network. The influence of production time is accounted for using dimension expansion. Our model directly predicts the remaining oil saturation distribution with known reservoir permeability maps and varying well schedules. We examine the performance of our model in 2D and 3D waterflooding reservoirs, which shows that the prediction results are in good consistence with the reservoir numerical simulation. The model can be extended to account for the impacts of other factors, such as initial oil saturation distribution, well patterns, and non-frequent heterogeneities. Although deep learning takes a longer time for dataset preparation in comparison to the traditional simulation, the prediction efficiency of our trained model is almost two orders of magnitude higher than that of numerical simulation. Therefore, this surrogate model can act as an efficient tool for automatic history matching and production optimization—which require multiple simulation runs.

Remote Sensing Image Semantic Segmentation Method Based on a Deep Convolutional Neural Network and Multiscale Feature Fusion

There are many problems with remote sensing images, such as large data scales, complex illumination conditions, occlusion, and dense targets. The existing semantic segmentation methods for remote sensing images are not accurate enough for small and irregular target segmentation results, and the edge extraction results are poor. The authors propose a remote sensing image segmentation method based on a DCNN and multiscale feature fusion. Firstly, an end-to-end remote sensing image segmentation model using complete residual connection and multiscale feature fusion was designed based on a deep convolutional encoder–decoder network. Secondly, weighted high-level features were obtained using an attention mechanism, which better preserved the edges, texture, and other information of remote sensing images. The experimental results on ISPRS Potsdam and Urban Drone datasets show that compared with the comparison methods, this method has better segmentation effect on small and irregular objects and achieves the best segmentation performance while ensuring the computation speed.

A novel deep convolutional encoder–decoder network: application to moving object detection in videos

A novel deep convolutional encoder–decoder network: application to moving object detection in videos

Urban scene segmentation model based on multi-scale shuffle features.

The monitoring of urban land categories is crucial for effective land resource management and urban planning. To address challenges such as uneven parcel distribution, difficulty in feature extraction and loss of image information in urban remote sensing images, this study proposes a multi-scale feature shuffle urban scene segmentation model. The model utilizes a deep convolutional encoder-decoder network with BlurPool instead of MaxPool to compensate for missing translation invariance. GSSConv and SE module are introduced to enhance information interaction and filter redundant information, minimizing category misclassification caused by similar feature distributions. To address unclear boundary information during feature extraction, the model applies multi-scale attention to aggregate context information for better integration of boundary and global information. Experiments conducted on the BDCI2017 public dataset show that the proposed model outperforms several established segmentation networks in OA, mIoU, mRecall, P and Dice with scores of 83.1%, 71.0%, 82.7%, 82.7% and 82.5%, respectively. By effectively improving the completeness and accuracy of urban scene segmentation, this study provides a better understanding of urban development and offers suggestions for future planning.

White Matter Perivascular Space Burden Is Associated With Mild Traumatic Brain Injury History In Soldiers

Mild traumatic brain injury (mTBI) exposure in military Service Members may lead to impaired brain waste clearance which increases neurological disease risk including amyloidopathies. Perivascular spaces (PVS) are part of the glymphatic pathway which support brain waste clearance, especially during sleep. Greater magnetic resonance imaging (MRI)-visible PVS burden has been previously observed in patients with neurodegenerative diseases and animal TBI models. PURPOSE: To determine the association between PVS burden (number and volume) and mTBI history in Special Operations Forces (SOF) combat Soldiers. METHODS: In this cross-sectional study, participants underwent neuroimaging to capture T2-weighted (T2w) whole brain three dimensional volumes obtained on a Siemens 3T Biograph mMR or 3T MAGNETOM Prisma scanner (TR/TE = 3200/400ms, slice thickness = 1mm, FoV = 256mm x 256 mm). The PVS segmentation was performed using a previously validated, multi-scale deep convolutional encoder-decoder neural network. All T2w images were skull stripped and a custom T2w atlas with segmented cerebral white matter and subcortical gray matter labels was registered to each subject using Advanced Normalization Tools to minimize misidentifying thin tubular structures in the skull, brainstem, cerebellum, and cortex as PVS. Only PVS clusters within the white matter mask were quantified for analyses. Binary mTBI history was self-reported (“yes” or “no”). Due to non-normal PVS metric distribution, non-parametric Mann-Whitney U tests were used to determine group differences in PVS outcomes. RESULTS: In total, 223 healthy SOF combat Soldiers (age = 33.1±4.3yrs) were included for analysis and 123 (55.16%) reported mTBI history. Soldiers with mTBI history had greater PVS number (z = 2.506, P = 0.013) and PVS volume (z = 2.419, P = 0.016). CONCLUSIONS: We found mTBI history is associated with increased PVS burden in SOF combat Soldiers that are clinically recovered from mTBI. This may indicate ongoing physiological post-mTBI changes that could lead to impaired waste clearance via the glymphatic system. Future studies should determine how sleep influences PVS burden following mTBI exposure and if PVS number and volume are meaningful neurobiological outcomes for neurodegenerative disease risk. Funded by US Special Operations Command

Perivascular Space Morphology Is Affected By Career Exposure In Special Operations Forces Combat Soldiers

Special Operations Forces (SOF) combat Soldiers are at high risk for repetitive blast and head impact exposure. These exposures may predispose SOF combat Soldiers to acute brain injuries and subsequent chronic manifestations including sleep disorders reported by 40% of US Service Members. Studying brain morphology such as perivascular spaces (PVS) may illustrate a strong intersection between brain glymphatic system function and sleep quality. Cerebral PVS are glymphatic system passageways normally serving to clear brain metabolic waste during sleep. Increasing MRI visible PVS number and volume are a possible pathogenesis for neurodegenerative disorders. PURPOSE: To determine the association between PVS morphological characteristics and career stage in SOF combat Soldiers. METHODS: Healthy asymptomatic SOF combat Soldiers (age=33.0±4.1yrs) cross-sectionally participated in this study at two career stages: baseline (n=141) and in-career (n=76). We collected whole brain 3D T2-weighted magnetic resonance imaging (MRI) on a Siemens 3T Biograph mMR or 3T MAGNETOM Prisma with equivalent settings (TR/TE = 3200/400ms, slice thickness = 1mm, FoV = 256mm x 256 mm). We segmented the PVS using validated multi-scale deep convolutional encoder-decoder neural networks. All images were skull stripped and registered to a custom brain atlas using Advanced Normalization Tools to identify only PVS clusters within cerebral white matter. We quantified PVS number and volume. The PVS outcome variables were non-normally distributed. As such, we employed nonparametric Mann Whitney U tests to compare the two career stage groups. RESULTS: Median (IQR) data were as follows: PVS number = 37.0 (44.0); PVS volume = 647mm3 (972mm3). In-career SOF combat Soldiers had greater PVS number (z = 2.56, P = 0.01) and PVS volume (z = 2.28, P = 0.02). CONCLUSIONS: In-career SOF combat Soldiers (typically older) show higher segmented PVS number and volume compared to baseline cohort. Also, our PVS findings may indicate impairments with clearing brain wastes suggesting sleep interventions may be warranted in SOF combat Soldier populations who experience continued exposures to combat and training. Measuring the PVS may be an important outcome in determining the efficacy of sleep studies in this population. Funded by US Special Operations Command.

Multi-Scale Identity-Preserving Image-to-Image Translation Network for Low-Resolution Face Recognition

State-of-the-art deep neural network models have reached near perfect face recognition accuracy rates on controlled high-resolution face images (especially Caucasian faces). However, their performance is drastically degraded when they are tested with very low-resolution face images. This is particularly critical in surveillance systems, where a low-resolution probe image is to be matched with high-resolution gallery images. Super-resolution techniques aim at producing high-resolution face images from low-resolution counterparts. While they are capable of reconstructing images that arevisually appealing, the identity-related information is not preserved. Here, we propose an identity-preserving end-to-end image-to-image translation deep neural network which is capable of super-resolving very low-resolution faces to their high-resolution counterparts while preserving identity-related information. We achieved this by training a very deep convolutional encoder-decoder network with a symmetric contracting path between corresponding layers. This network was trained with a combination of a reconstruction andan identity-preserving loss, on multi-scale low-resolution conditions. Extensive quantitative evaluations of our proposed model demonstrated that it outperforms competing super-resolution and low-resolution face recognition methods on natural and artificial low-resolution face data sets and even unseen identities.

Learned prior-guided algorithm for flow field visualization in electrical capacitance tomography

Electrical capacitance tomography provides great potential advantages in measuring flow field parameters by providing information about spatial-temporal medium distributions, but it is plagued by low-quality reconstructions. In order to overcome this challenge, in this work, the learned prior (LP) that bridges the measurement physics and data-driven modeling paradigms is introduced and coupled with the measurement physics and the domain knowledge into a novel imaging model for reshaping the tomographic reconstruction problem. The LP captures spatial details of imaging objects and guides the search to discover high-quality solutions. A new multi-fidelity deep learning is developed to predict the LP by using deep convolutional encoder-decoder network and multi-fidelity samples, which reduces the difficulty and cost of collecting high-fidelity samples. The established imaging model is solved within the framework of the half-quadratic optimization method. This work transforms the image reconstruction paradigm by fusing the measurement physics and data-driven modeling paradigms. The assessment results have validated that this novel method provides a range of advantages over popular methods, including higher reconstruction quality (RQ) and better robustness.

Joint segmentation of optic cup and optic disc using deep convolutional generative adversarial network

Glaucoma, as one of the three major blinding ophthalmic diseases in the world, is usually accompanied by changes in the structure of the patient’s optic disc, such as optic disc atrophy and depression. Clinical ophthalmologists tend to use the cup-disc ratio as an evaluation index to realize the screening and diagnosis of glaucoma. Therefore, the accurate measurement of optic cup (OC), optic disc (OD) and other parameters is of great clinical significance for early screening of glaucoma. Inspired by game theory, this paper combines deep convolutional neural networks (DCNN) with generative adversarial networks (GAN), and proposes a model for the joint segmentation of OC and OD. Specifically, the generator is composed of a deep convolutional encoder-decoder network to jointly segment the OC and OD, and the discriminator is composed of an eight layer full convolutional neural network. The discrimination results adjust the parameters in the structure of the generator and discriminator network through back propagation to achieve the effect of autonomous learning and optimization of the model. When the proposed network and the existing networks are evaluated on the public dataset Drishti-GS1, the research results demonstrate that the proposed network can achieve a significant improvement in the overall performance.

Automated food intake tracking requires depth-refined semantic segmentation to rectify visual-volume discordance in long-term care homes

Malnutrition is a multidomain problem affecting 54% of older adults in long-term care (LTC). Monitoring nutritional intake in LTC is laborious and subjective, limiting clinical inference capabilities. Recent advances in automatic image-based food estimation have not yet been evaluated in LTC settings. Here, we describe a fully automatic imaging system for quantifying food intake. We propose a novel deep convolutional encoder-decoder food network with depth-refinement (EDFN-D) using an RGB-D camera for quantifying a plate’s remaining food volume relative to reference portions in whole and modified texture foods. We trained and validated the network on the pre-labelled UNIMIB2016 food dataset and tested on our two novel LTC-inspired plate datasets (689 plate images, 36 unique foods). EDFN-D performed comparably to depth-refined graph cut on IOU (0.879 vs. 0.887), with intake errors well below typical 50% (mean percent intake error: -4.2%). We identify how standard segmentation metrics are insufficient due to visual-volume discordance, and include volume disparity analysis to facilitate system trust. This system provides improved transparency, approximates human assessors with enhanced objectivity, accuracy, and precision while avoiding hefty semi-automatic method time requirements. This may help address short-comings currently limiting utility of automated early malnutrition detection in resource-constrained LTC and hospital settings.

Deep Convolutional Symmetric Encoder—Decoder Neural Networks to Predict Students’ Visual Attention

Prediction of visual attention is a new and challenging subject, and to the best of our knowledge, there are not many pieces of research devoted to the anticipation of students’ cognition when solving tests. The aim of this paper is to propose, implement, and evaluate a machine learning method that is capable of predicting saliency maps of students who participate in a learning task in the form of quizzes based on quiz questionnaire images. Our proposal utilizes several deep encoder–decoder symmetric schemas which are trained on a large set of saliency maps generated with eye tracking technology. Eye tracking data were acquired from students, who solved various tasks in the sciences and natural sciences (computer science, mathematics, physics, and biology). The proposed deep convolutional encoder–decoder network is capable of producing accurate predictions of students’ visual attention when solving quizzes. Our evaluation showed that predictions are moderately positively correlated with actual data with a coefficient of 0.547 ± 0.109. It achieved better results in terms of correlation with real saliency maps than state-of-the-art methods. Visual analyses of the saliency maps obtained also correspond with our experience and expectations in this field. Both source codes and data from our research can be downloaded in order to reproduce our results.

Blind microscopy image denoising with a deep residual and multiscale encoder/decoder network.

In computer-aided diagnosis (CAD) focused on microscopy, denoising improves the quality of image analysis. In general, the accuracy of this process may depend both on the experience of the microscopist and on the equipment sensitivity and specificity. A medical image could be corrupted by several perturbations during image acquisition. Nowadays, CAD deep learning applications pre-process images with image denoising models to reinforce learning and prediction. In this work, an innovative and lightweight deep multiscale convolutional encoder-decoder neural network is proposed. Specifically, the encoder uses deterministic mapping to map features into a hidden representation. Then, the latent representation is rebuilt to generate the reconstructed denoised image. Residual learning strategies are used to improve and accelerate the training process using skip connections in bridging across convolutional and deconvolutional layers. The proposed model reaches on average 38.38 of PSNR and 0.98 of SSIM on a test set of 57458 images overcoming state-of-the-art models in the same application domain.Clinical relevance - Encoder-decoder based denoiser enables industry experts to provide more accurate and reliable medical interpretation and diagnosis in a variety of fields, from microscopy to surgery, with the benefit of real-time processing.

Automatic Classification of Pararubber Trees in Thailand from LANDSAT-8 Images Using Neural Network Method

Classifying features from satellite images has been a time-consuming manual process which requires lots of manpower. This work exploits deep convolutional encoder-decoder neural network to develop an algorithm that can automatically classify the extents of the Pararubber tree-growing areas from the LANDSAT-8 images. The ground truth of the areas of the Pararubber tree was manually prepared and was separated into training datasets and the validation datasets. The classification model from this approach obtained using the training datasets was verified with the classification accuracy of70.90%, precision of 67.66%, recall of 80.80%, and F1 score of 73.59%.

Microaneurysm detection in fundus images based on a novel end-to-end convolutional neural network

Microaneurysms are the earliest symptom of diabetic retinopathy and play an important role in the screening of diabetic retinopathy. However, because of the complex background, automatic detection microaneurysm in fundus images is a challenging task. Firstly, motivated by the characteristics of microaneurysm, a novel deep convolutional encoder-decoder network for microaneurysm detection is designed to locate the MAs by the differences between the skip connection in the network. Then, a weighted dice loss, termed the smooth dice loss, is presented to put more focus on misclassified microaneurysms. Finally, an activation function with a long tail is used to produce an accurate probability map for MA detection. Plenty of experiments, conducted on the Retinopathy Online Challenge dataset and the e-ophtha-MA dataset, demonstrate that the proposed model achieves the comparable performance to the existing state-of-the-art methods on microaneurysm detection with only one-hundredth the running time compared with its counterparts. The proposed method is simple and effective, guarantees the performance while shortening the test time. It indicates the potential application in the auxiliary diagnosis of diabetic retinopathy screening.

DeepCEDNet: An Efficient Deep Convolutional Encoder-Decoder Networks for ECG Signal Enhancement

Electrocardiogram (ECG) signal can be thought of as an effective indicator for detection of various arrhythmias. However, the acquired ECG data is always corrupted by amounts of noise, which have a great influence on the diagnosis of cardiovascular diseases. In this paper, an efficient deep convolutional encoder-decoder network framework is proposed to remove the noise from ECG signal, which is termed as ‘DeepCEDNet’. This network is able to learn a sparse representation of data in the time-frequency domain via the high-order synchrosqueezing transform (FSSTH) and a nonlinear function that maps the noisy data into the clean one based on the distribution difference between signal and noise from the training set. Extensive experiments are conducted on ECG signals from the MIT-BIH Arrhythmia database and MIT-BIH Long-Term ECG database, and the added noise is obtained from the MIT-BIH Noise Stress Test database. The denoising performance is evaluated by means of signal to noise ratio (SNR), root mean squared error (RMSE) and percent root mean square difference (PRD). The results indicate that the proposed DeepCEDNet can obtain superior performance in both noise reduction and details preservation with higher SNR and lower RMSE and PRD compared to the traditional convolutional neural network (CNN) and the fully convolutional network-based denoising auto-encoder (FCN). We believe that the DeepCEDNet has a wide application prospect in the biomedical field.

Semantic Segmentation of Anaemic RBCs Using Multilevel Deep Convolutional Encoder-Decoder Network

Pixel-level analysis of blood images plays a pivotal role in diagnosing blood-related diseases, especially Anaemia. These analyses mainly rely on an accurate diagnosis of morphological deformities like shape, size, and precise pixel counting. In traditional segmentation approaches, instance or object-based approaches have been adopted that are not feasible for pixel-level analysis. The convolutional neural network (CNN) model required a large dataset with detailed pixel-level information for the semantic segmentation of red blood cells in the deep learning domain. In current research work, we address these problems by proposing a multi-level deep convolutional encoder-decoder network along with two state-of-the-art healthy and Anaemic-RBC datasets. The proposed multi-level CNN model preserved pixel-level semantic information extracted in one layer and then passed to the next layer to choose relevant features. This phenomenon helps to precise pixel-level counting of healthy and anaemic-RBC elements along with morphological analysis. For experimental purposes, we proposed two state-of-the-art RBC datasets, i.e., Healthy-RBCs and Anaemic-RBCs dataset. Each dataset contains 1000 images, ground truth masks, relevant, complete blood count (CBC), and morphology reports for performance evaluation. The proposed model results were evaluated using crossmatch analysis with ground truth mask by finding IoU, individual training, validation, testing accuracies, and global accuracies using a 05-fold training procedure. This model got training, validation, and testing accuracies as 0.9856, 0.9760, and 0.9720 on the Healthy-RBC dataset and 0.9736, 0.9696, and 0.9591 on an Anaemic-RBC dataset. The IoU and BFScore of the proposed model were 0.9311, 0.9138, and 0.9032, 0.8978 on healthy and anaemic datasets, respectively.

Skin Lesion Segmentation Using Local Binary Convolution-Deconvolution Architecture

Deep learning techniques such as Deep Convolutional Networks have achieved great success in skin lesion segmentation towards melanoma detection. The performance is however restrained by distinctive and challenging features of skin lesions such as irregular and fuzzy border, noise and artefacts presence and low contrast between lesions. The methods are also restricted with scarcity of annotated lesion images training dataset and limited computing resources. Recent research in convolutional neural network (CNN) has provided a variety of new architectures for deep learning. One interesting new architecture is the local binary convolutional neural network (LBCNN), which can reduce the workload of CNNs and improve the classification accuracy. The proposed framework employs the local binary convolution on U-net architecture instead of the standard convolution in order to reduced-size deep convolutional encoder-decoder network that adopts loss function for robust segmentation. The proposed framework replaced the encoder part in U-net by LBCNN layers. The approach automatically learns and segments complex features of skin lesion images. The encoder stage learns the contextual information by extracting discriminative features while the decoder stage captures the lesion boundaries of the skin images. This addresses the issues with encoder-decoder network producing coarse segmented output with challenging skin lesions appearances such as low contrast between healthy and unhealthy tissues and fine grained variability. It also addresses issues with multi-size, multi-scale and multi-resolution skin lesion images. The deep convolutional network also adopts a reduced-size network with just five levels of encoding-decoding network. This reduces greatly the consumption of computational processing resources. The system was evaluated on publicly available dataset of ISIC and PH2. The proposed system outperforms most of the existing state-of-art.

Adaptive Thermal Sensor Array Placement for Human Segmentation and Occupancy Estimation

Thermal sensor array (TSA) offers privacy-preserving, low-cost, and non-invasive features, which makes it suitable for various indoor applications such as anomaly detection, health monitoring, home security, and monitoring energy efficiency applications. Previous approaches to human-centred applications using the TSA usually relied on the use of a fixed sensor location to make the human-sensor distance and the human presence shape fixed. However, placing this sensor in different locations and new indoor environments can pose a significant challenge. In this paper, a novel framework based on a deep convolutional encoder-decoder network is proposed to address this challenge in real-life deployment. The framework presents a semantic segmentation of the human presence and estimates the occupancy in indoor-environment. It is also capable to segment the human presence and counts the number of people from different sensor locations, indoor environments, and human to sensor distance. Furthermore, the impact of the distance on the human presence using the TSA is investigated. The framework is evaluated to estimate the occupancy in different sensor locations, the number of occupants, environments, and human distance with classification and regression machine learning approaches. This paper shows that the classification approach using the adaptive boosting algorithm is an accurate approach which has achieves an accuracy of 98.43% and 100% from vertical and overhead sensor locations respectively.