Deep Convolutional Encoder Decoder Research Articles

Crop pest identification using deep network based extracted features and MobileENet in smart agriculture

AbstractAgriculture has been considered an important source of food for humans throughout history. Plant pests cause significant damage to crops and reduce the productivity of global crop yields. Therefore, it is important to identify the plant pest at an earlier stage in order to minimize crop losses and use pesticides optimally. This paper develops the MobileENet deep learning architecture for accurate plant pest identification with less computational effort. The input images are pre‐processed, and the features are extracted using a deep convolutional encoder–decoder network (DCEDN). The proposed classification approach solves the problems of over‐fitting regularization, batch normalization, and dropout layers. Due to the minimum computing size and factorization process, the classification performance is increased. It extracts discriminatory feature information by eliminating redundant background information. The performance of the proposed approach is evaluated on the IP102 dataset, and the performance is compared with existing deep learning‐based approaches. The performance metrics, such as accuracy, precision, recall, and so forth, are considered to evaluate the performance of the proposed plant pest identification approach. The accuracy performance of the proposed approach is improved to 98.83% with less information loss.

Deep convolutional encoder–decoder networks based on ensemble learning for semantic segmentation of high-resolution aerial imagery

Deep convolutional encoder–decoder networks based on ensemble learning for semantic segmentation of high-resolution aerial imagery

Predicting Time-to-Healing from a Digital Wound Image: A Hybrid Neural Network and Decision Tree Approach Improves Performance

Despite the societal burden of chronic wounds and despite advances in image processing, automated image-based prediction of wound prognosis is not yet in routine clinical practice. While specific tissue types are known to be positive or negative prognostic indicators, image-based wound healing prediction systems that have been demonstrated to date do not (1) use information about the proportions of tissue types within the wound and (2) predict time-to-healing (most predict categorical clinical labels). In this work, we analyzed a unique dataset of time-series images of healing wounds from a controlled study in dogs, as well as human wound images that are annotated for the tissue type composition. In the context of a hybrid-learning approach (neural network segmentation and decision tree regression) for the image-based prediction of time-to-healing, we tested whether explicitly incorporating tissue type-derived features into the model would improve the accuracy for time-to-healing prediction versus not including such features. We tested four deep convolutional encoder–decoder neural network models for wound image segmentation and identified, in the context of both original wound images and an augmented wound image-set, that a SegNet-type network trained on an augmented image set has best segmentation performance. Furthermore, using three different regression algorithms, we evaluated models for predicting wound time-to-healing using features extracted from the four best-performing segmentation models. We found that XGBoost regression using features that are (i) extracted from a SegNet-type network and (ii) reduced using principal components analysis performed the best for time-to-healing prediction. We demonstrated that a neural network model can classify the regions of a wound image as one of four tissue types, and demonstrated that adding features derived from the superpixel classifier improves the performance for healing-time prediction.

A Deep Convolutional Encoder–Decoder–Restorer Architecture for Image Deblurring

The accuracy of many computer vision tasks is reduced by blurred images, so deblur is important. More details of the image can be captured by a common multi-stage network, but the computational complexity of this method is higher compared with a single-stage network. However, a single-stage network cannot capture multi-scale information well. To tackle the problem, a novel convolutional encoder–decoder–restorer architecture is proposed. In this architecture, a multi-scale input structure is used in the encoder. Improved supervised attention module is inserted into the encoder for enhanced feature acquisition. In decoder, information supplement block is proposed to fuse multi-scale features. Finally, the fused features are used for image recovery in the restorer. In order to optimise the model in multiple domains, the loss function is calculated separately in the spatial and frequency domains. Our method is compared with existing methods on the GOPRO dataset. In addition, to verify the applications of our proposed method, we conduct experiments on the Real image dataset, the VOC2007 dataset and the LFW dataset. Experimental results show that our proposed method outperforms state-of-the-art deblurring methods and improves the accuracy of different vision tasks.

DSLSTM: a deep convolutional encoder–decoder architecture for abnormality detection in video surveillance

DSLSTM: a deep convolutional encoder–decoder architecture for abnormality detection in video surveillance

A Physics-Informed Spatial-Temporal Neural Network for Reservoir Simulation and Uncertainty Quantification

Summary Surrogate models play a vital role in reducing computational complexity and time burden for reservoir simulations. However, traditional surrogate models suffer from limitations in autonomous temporal information learning and restrictions in generalization potential, which is due to a lack of integration with physical knowledge. In response to these challenges, a physics-informed spatial-temporal neural network (PI-STNN) is proposed in this work, which incorporates flow theory into the loss function and uniquely integrates a deep convolutional encoder-decoder (DCED) with a convolutional long short-term memory (ConvLSTM) network. To demonstrate the robustness and generalization capabilities of the PI-STNN model, its performance was compared against both a purely data-driven model with the same neural network architecture and the renowned Fourier neural operator (FNO) in a comprehensive analysis. Besides, by adopting a transfer learning strategy, the trained PI-STNN model was adapted to the fractured flow fields to investigate the impact of natural fractures on its prediction accuracy. The results indicate that the PI-STNN not only excels in comparison with the purely data-driven model but also demonstrates a competitive edge over the FNO in reservoir simulation. Especially in strongly heterogeneous flow fields with fractures, the PI-STNN can still maintain high prediction accuracy. Building on this prediction accuracy, the PI-STNN model further offers a distinct advantage in efficiently performing uncertainty quantification, enabling rapid and comprehensive analysis of investment decisions in oil and gas development.

A deep learning based surrogate model for reservoir dynamic performance prediction

Predicting the reservoir dynamic performance is an important basis for exploiting the remaining oil and improving the recovery. Traditional prediction techniques primarily include numerical simulation and reservoir engineering methods, which show technical drawbacks such as low computational efficiency and too simplified assumptions. To tackle this challenge, we propose a surrogate model for reservoir dynamic performance prediction based on a Variety of View deep convolutional encoder-decoder (VoV-DCED) network. The influence of production time is accounted for using dimension expansion. Our model directly predicts the remaining oil saturation distribution with known reservoir permeability maps and varying well schedules. We examine the performance of our model in 2D and 3D waterflooding reservoirs, which shows that the prediction results are in good consistence with the reservoir numerical simulation. The model can be extended to account for the impacts of other factors, such as initial oil saturation distribution, well patterns, and non-frequent heterogeneities. Although deep learning takes a longer time for dataset preparation in comparison to the traditional simulation, the prediction efficiency of our trained model is almost two orders of magnitude higher than that of numerical simulation. Therefore, this surrogate model can act as an efficient tool for automatic history matching and production optimization—which require multiple simulation runs.

Remote Sensing Image Semantic Segmentation Method Based on a Deep Convolutional Neural Network and Multiscale Feature Fusion

There are many problems with remote sensing images, such as large data scales, complex illumination conditions, occlusion, and dense targets. The existing semantic segmentation methods for remote sensing images are not accurate enough for small and irregular target segmentation results, and the edge extraction results are poor. The authors propose a remote sensing image segmentation method based on a DCNN and multiscale feature fusion. Firstly, an end-to-end remote sensing image segmentation model using complete residual connection and multiscale feature fusion was designed based on a deep convolutional encoder–decoder network. Secondly, weighted high-level features were obtained using an attention mechanism, which better preserved the edges, texture, and other information of remote sensing images. The experimental results on ISPRS Potsdam and Urban Drone datasets show that compared with the comparison methods, this method has better segmentation effect on small and irregular objects and achieves the best segmentation performance while ensuring the computation speed.

Comparison of deep learning models to extract silt storage dams from remote sensing images

Determining the locations and shapes of silt storage dams (SSDs) is necessary before planning and constructing new ones or maintaining old ones. Google images with a spatial resolution of 0.54 m were cropped, labelled and enhanced to establish two schemes of remote sensing images that contain SSDs with different input and batch sizes. Five deep learning models (FCN (fully connected convolutional neural network, SegNet (deep convolutional encoder–decoder architecture for image segmentation), U-Net (convolutional networks for biomedical image segmentation), PSPNet (pyramid scene parsing network) and DeepLab-V3+) were constructed to extract SSDs from the images based on the two schemes. The loss curves, accuracies and extraction results derived from the five models were compared to identify the optimal model for SSD extraction. The results show that the overall accuracies, F1 scores and mean intersections over unions obtained from DeepLab-V3+ were, respectively, 95.29%, 70.33% and 74.13% for scheme 1 (S1) and 96.29%, 73.34% and 76.99% for scheme 2 (S2), which were better than the values for other models. PSPNet had the shortest training times (128 s/step for S1 and 348 s/step for S2). An input size of 480 × 480 pixels, a batch size of 4 and 2304 images enhanced the extraction accuracy and prevented overfitting. The results provide a reference for the planning, construction and maintenance of water and soil conservation measures.

A novel deep convolutional encoder–decoder network: application to moving object detection in videos

A novel deep convolutional encoder–decoder network: application to moving object detection in videos

Geometric consistency enhanced deep convolutional encoder-decoder for urban seismic damage assessment by UAV images

Precise and rapid assessment of seismic damage to buildings is critical for urban regions. To address this challenge, this study proposes QuakeCityNet (QCNet-M-N)- a model with flexible configurations of M encoding stages and N embedding convolution operations for exact pixel-level recognition of earthquake-damaged buildings using unmanned aerial vehicle (UAV) images. A novel loss function, geometric consistency enhanced (GCE) loss, is designed to focus on the building regions and local boundaries, taking into account the geometrical constraints of split line length, curvature, and area. Test results indicate that the proposed QCNet model can achieve robust and stable segmentation accuracy under diverse weather conditions, such as abnormal illumination, rain, and fog. Moreover, the utilization of GCE loss significantly reduces the false-positive small-region noise while preserving overall geometrical shapes. Finally, an application of seismic assessment is conducted in Beichuan county to demonstrate the effectiveness of the proposed method.

Solving Electromagnetic Inverse Scattering Problems in Inhomogeneous Media by Deep Convolutional Encoder–Decoder Structure

This communication proposes a novel deep learning (DL) approach to solve electromagnetic inverse scattering (EMIS) problems in inhomogeneous media. The conventional approaches for solving inhomogeneous EMIS problems generally have to consider inhomogeneous Green’s functions or conduct approximation operations to media, which inevitably introduces various challenges, including complex mathematical derivation, high computation cost, unavoidable nonlinearity, and even strong ill-posedness. To avoid these challenges, we propose a DL approach based on the complex-valued deep convolutional neural networks (DConvNets), which comprise the deep convolutional encoder–decoder (DCED) structure. Its training data are collected based on the simple synthetic dataset. While the scattered fields received in the measurement domain are utilized as the input for the encoder to extract feature fragments, the final output for the counterpart decoder is the contrasts (permittivities) of dielectric scatterers in the target domain. In this way, unlike the conventional methods, the unknown domain between the target domain and measurement domain never has to be considered to compute inhomogeneous Green’s functions. Consequently, the inhomogeneous EMIS problems could be solved with higher accuracy even for extremely high-contrast targets. Numerical examples illustrate the feasibility of the proposed DL approach. It acts as a new candidate for solving EMIS problems in inhomogeneous media.

Enhanced Two-Step Deep-Learning Approach for Electromagnetic-Inverse-Scattering Problems: Frequency Extrapolation and Scatterer Reconstruction

The electromagnetic-inverse-scattering (EMIS) problem is solved by a novel two-step deep-learning (DL) approach in this article. The newly proposed two-step DL approach not only predicts the multifrequency EM scattered field, but also overcomes the limitation of the conventional methods for solving EMIS problems, such as expensive computational cost, strong ill-conditions, and invalidity on high contrast. In the first step, the complex-valued deep residual convolutional neural network (DRCNN) is utilized to predict multifrequency EM scattered fields only using single-frequency EM scattered field information. Based on a new complex-valued deep convolutional encoder-decoder (DCED) structure, the second step utilizes the obtained multifrequency EM scattered field “images” to realize the reconstruction of the target scatterers. In such a manner, the proposed approach can solve the EMIS problem accurately and efficiently even for inhomogeneous and high-contrast scatterers. The training of the proposed two DL models is based on the simple synthetic dataset. Numerical examples based on various dielectric objects are given to demonstrate the accuracy and performance of the newly proposed approach. The proposed DL-based method opens a new path for handling real-time quantitative microwave imaging.

Elastic Full-Waveform Inversion Using a Physics-Guided Deep Convolutional Encoder–Decoder

Elastic Full-Waveform Inversion Using a Physics-Guided Deep Convolutional Encoder–Decoder

Modelling long-term thermal comfort conditions in urban environments using a deep convolutional encoder-decoder as a computational shortcut

Different urban microscale models exist to model street-level mean radiation temperature (Tmrt). However, these models are computationally expensive, albeit to varying degrees. We present a computational shortcut using a convolutional encoder-decoder network (U-Net) to predict pedestrian level (1.1 m a.g.l.) Tmrt at a building-resolved scale (1 × 1 m). SOLWEIG is used to create spatial training data for 68 days at hourly resolution in the city of Freiburg, Germany. Validation of the model was carried out in two steps: First, SOLWEIG (and U-Net) were validated against Tmrt point measurements. Second, U-Net was validated against SOLWEIG on 6 areas and 12 days resulting in a MAE of 2.4 K. The U-Net is 22 times faster than SOLWEIG, and thus able to emulate a micrometeorological physical model with computational superiority. As a demonstration case, U-Net is applied to model Tmrt for the urbanized area of Freiburg for two complete 30-year periods (1961–1990, 1991–2020) driven by hourly ERA5-Land reanalysis data. Summertime daily maximum Tmrt increased on average by 2.5 K, whereas summertime daily maximum air temperature increased by only 1.5 K. Maximum Tmrt increase is stronger on non-tree covered paved areas (2.8 K) than on tree covered grassy areas (1.8 K).

Enhancing the accuracy of shock advisory algorithms in automated external defibrillators during ongoing cardiopulmonary resuscitation using a deep convolutional Encoder-Decoder filtering model

Survival from out-of-hospital cardiac arrests (OHCA) depends on an accurate defibrillatory shock decision during cardiopulmonary resuscitation (CPR). Since chest compressions induce severe motion artifact in the electrocardiogram (ECG), current automatic external defibrillators (AEDs) instruct the user not to perform CPR during the rhythm analysis period. However, performing continuous CPR is vital and dramatically increases the chance of survival. Hence, we demonstrate a novel application of a deep convolutional neural network encoder-decoder (CNNED) method to suppress CPR artifact in near real-time, using only ECG data. The encoder portion of the CNNED uses the magnitude and phase contents derived via time-varying spectral analysis to learn distinct features that are representative of both the ECG signal and CPR artifact. The decoder portion takes the results from the encoder and reconstructs what is perceived as the motion artifact-removed ECG data. These procedures are done via a multitude of training of the CNNED using many different arrhythmias contaminated with CPR. In this study, CPR-contaminated ECGs were generated by combining clean ECGs with 52 different CPR artifacts. ECG data from CUDB, VFDB, SDDB, and AFDB datasets which belong to the Physionet’s Physiobank archive were used to create the training set containing 14-second ECG segments. The performance of the proposed CNNED was evaluated on a separate test set comprised of 23,816 CPR-contaminated 14-second ECG segments from 458 subjects. The results were evaluated by three metrics: signal-to-noise ratio (SNR), correlation coefficient, and accuracy of Defibtech’s AED shock advisory algorithm (SAA). CNNED resulted in an increase of the mean SNR value from −3 dB to 7.5 dB and 7.1 dB for shockable and non-shockable rhythms, respectively. 80.85% of the filtered shockable and 74.13% of the filtered non-shockable ECG data were highly correlated (>0.7) with the artifact-free ECG; these values were only 13.30% and 12.71% for CPR-contaminated shockable and non-shockable, respectively, without our filtering approach. Comparing results of Defibtech’s AED SAA before and after applying CNNED on the CPR-contaminated ECG, the specificity improved from 96.21% to 99.14% for normal sinus rhythm, and from 88.5% to 96.45% for other non-shockable rhythms. The sensitivity of shockable detection also increased from 67.68% to 90.90% for ventricular fibrillation, and from 62.71% to 82.26% for ventricular tachycardia. These results indicate continuous and accurate AED rhythm analysis without stoppage of CPR, using only ECG data.

SUnCNN: Sparse Unmixing Using Unsupervised Convolutional Neural Network

In this letter, we propose a sparse unmixing technique using a convolutional neural network (SUnCNN) for hyperspectral images. SUnCNN is the first deep learning-based technique proposed for sparse unmixing. It uses a deep convolutional encoder–decoder to generate the abundances relying on a spectral library. We reformulate the sparse unmixing into an optimization over the deep network’s parameters. Therefore, the deep network learns in an unsupervised manner to map a fixed input into the sparse optimum abundances. Additionally, SUnCNN holds the sum-to-one constraint using a softmax activation layer. The proposed method is compared with the state-of-the-art using two synthetic datasets and one real hyperspectral dataset. The overall results confirm that the proposed method outperforms the other ones in terms of signal to reconstruction error (SRE). Additionally, SUnCNN shows visual superiority for both real and synthetic datasets compared with the competing techniques. The proposed method was implemented in Python (3.8) using PyTorch as the platform for the deep network and is available online: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/BehnoodRasti/SUnCNN</uri> .

Deep Convolutional Symmetric Encoder—Decoder Neural Networks to Predict Students’ Visual Attention

Prediction of visual attention is a new and challenging subject, and to the best of our knowledge, there are not many pieces of research devoted to the anticipation of students’ cognition when solving tests. The aim of this paper is to propose, implement, and evaluate a machine learning method that is capable of predicting saliency maps of students who participate in a learning task in the form of quizzes based on quiz questionnaire images. Our proposal utilizes several deep encoder–decoder symmetric schemas which are trained on a large set of saliency maps generated with eye tracking technology. Eye tracking data were acquired from students, who solved various tasks in the sciences and natural sciences (computer science, mathematics, physics, and biology). The proposed deep convolutional encoder–decoder network is capable of producing accurate predictions of students’ visual attention when solving quizzes. Our evaluation showed that predictions are moderately positively correlated with actual data with a coefficient of 0.547 ± 0.109. It achieved better results in terms of correlation with real saliency maps than state-of-the-art methods. Visual analyses of the saliency maps obtained also correspond with our experience and expectations in this field. Both source codes and data from our research can be downloaded in order to reproduce our results.

Frequency-Based Haze and Rain Removal Network (FHRR-Net) with Deep Convolutional Encoder-Decoder

Removing haze or rain is one of the difficult problems in computer vision applications. On real-world road images, haze and rain often occur together, but traditional methods cannot solve this imaging problem. To address rain and haze problems simultaneously, we present a robust network-based framework consisting of three steps: image decomposition using guided filters, a frequency-based haze and rain removal network (FHRR-Net), and image restoration based on an atmospheric scattering model using predicted transmission maps and predicted rain-removed images. We demonstrate FHRR-Net’s capabilities with synthesized and real-world road images. Experimental results show that our trained framework has superior performance on synthesized and real-world road test images compared with state-of-the-art methods. We use PSNR (peak signal-to-noise) and SSIM (structural similarity index) indicators to evaluate our model quantitatively, showing that our methods have the highest PSNR and SSIM values. Furthermore, we demonstrate through experiments that our method is useful in real-world vision applications.

Sequential Fault Detection and Classification in Chemical Process Using a Deep Convolutional Encoder-Decoder Architecture Based on System Dynamics

In this paper, we propose a high-level spatiotemporal feature extraction based on deep convolutional bidirectional encoder-decoder representation network with gated recurrent unit (GRU) cell for dynamic fault diagnosis. Although traditional recurrent neural networks (RNNs) have a major issue with extracting the spatial dependencies, temporal convolutional operation is designed to extract the local features which provide insight into different types of faults. The system dynamics are further extracted with the context by the bidirectional encoder-decoder network, which distils the representations from future time steps. Moreover, GRU cell in both encoder and decoder deploys a more compact structure than a usual RNNs network due to the reduction of gates. The resulting deep network is not only generalizing the importance of temporal feature, but it also allows the interpretable feature representation and classification simultaneously. Detailed comparative case study on the benchmark Tennessee Eastman process demonstrates that the proposed method is competitive with the classical long-short term memory (LSTM) network for the fault detection and diagnosis.