Encoder-decoder Network Research Articles

A two-branch encoder-decoder network for image tampering localization

Tampered images with false information can mislead viewers and pose security issues. Tampering traces in images are difficult to detect. To locate tampering traces effectively, a dual-domain deep-learning-based image tampering localization method based on RGB and frequency stream branches is proposed in this work. The former branch learns and extracts tampered features on the image and content features of the tampered region. The latter branch extracts tampered features from the frequency domain to complement the RGB stream branch. In addition, an attention mechanism is used to integrate the features from both branches at the fusion stage. In the experiments, the F1 score of the proposed method outperformed those of the baselines on the NIST16 dataset (with a 15.3 % improvement), and the AUC score outperformed those of the baselines on the NIST16 and COVERAGE datasets (improvements of 3.9 % and 4.7 %, respectively). This study provides a beneficial alternative to image tampering localization techniques.

ECGVEDNET: A Variational Encoder-Decoder Network for ECG Delineation in Morphology Variant ECGs.

Electrocardiogram (ECG) delineation to identify the fiducial points of ECG segments, plays an important role in cardiovascular diagnosis and care. Whilst deep delineation frameworks have been deployed within the literature, several factors still hinder their development: (a) data availability: the capacity of deep learning models to generalise is limited by the amount of available data; (b) morphology variations: ECG complexes vary, even within the same person, which degrades the performance of conventional deep learning models. To address these concerns, we present a large-scale 12-leads ECG dataset, ICDIRS, to train and evaluate a novel deep delineation model-ECGVEDNET. ICDIRS is a large-scale ECG dataset with 156,145 QRS onset annotations and 156,145 T peak annotations. ECGVEDNET is a novel variational encoder-decoder network designed to address morphology variations. In ECGVEDNET, we construct a well-regularized latent space, in which the latent features of ECG follow a regular distribution and present smaller morphology variations than in the raw data space. Finally, a transfer learning framework is proposed to transfer the knowledge learned on ICDIRS to smaller datasets. On ICDIRS, ECGVEDNET achieves accuracy of 86.28%/88.31% within 5/10 ms tolerance for QRS onset and accuracy of 89.94%/91.16% within 5/10 ms tolerance for T peak. On QTDB, the average time errors computed for QRS onset and T peak are -1.86 ± 8.02 ms and -0.50 ± 12.96 ms, respectively, achieving state-of-the-art performances on both large and small-scale datasets. We will release the source code and the pre-trained model on ICDIRS once accepted.

Variational Autoencoders for Generative Drug-Gene Interactions in Periodontal Bone Resorption.

Introduction Periodontal bone resorption is a significant dental problem causing tooth loss and impaired oral function. It is influenced by factors such as bacterial plaque, genetic predisposition, smoking, systemic diseases, medications, hormonal changes, and poor oral hygiene. This condition disrupts bone remodeling, favoring resorptive processes. Variational autoencoders (VAEs) can learn the distribution of drug-gene interactions from existing data, identify potential drug targets, and predict therapeutic effects. This study investigates the generation of drug-gene interactions in periodontal bone resorption using VAEs. Methods A bone resorptive drugs dataset was retrieved from Probes and Drugs and analyzed using Cytoscape (https://cytoscape.org/) and CytoHubba (https://apps.cytoscape.org/apps/cytohubba), powerful tools for studying drug-gene interactions in bone resorption. The dataset was then prepared for matrix representation, with normalized input data. It was subsequently divided into training, validation, and testing sets. We then built an encoder-decoder network, defined a loss function, optimized parameters, and fine-tuned hyperparameters. Using VAEs, we generated new drug-gene interactions, assessed model performance, and visualized the latent space with reconstructed drug-gene interactions for further insights. Results The analysis revealed the top hub genes in drug-gene interactions, including Matrix Metalloproteinase (MMP) 14, MMP 9, HIF1A, STAT1, MAPT, CAS9, MMP2, CASP3, MMP1, and MAK1. The VAE's reconstruction accuracy was measured using mean squared error (MSE), with an average squared difference of 0.077. Additionally, the KL divergence value was 2.349, and the average reconstruction log-likelihood was -246. Conclusion The generative variational encoder model for drug-gene interactions in bone resorption demonstrates high accuracy and reliability in representing complex drug-gene relationships within this context.

Ischemic stroke infarct segmentation model based on depthwise separable convolution for multimodal magnetic resonance imaging

Magnetic resonance imaging (MRI) plays a crucial role in the diagnosis of ischemic stroke. Accurate segmentation of the infarct is of great significance for selecting intervention treatment methods and evaluating the prognosis of patients. To address the issue of poor segmentation accuracy of existing methods for multiscale stroke lesions, a novel encoder-decoder architecture network based on depthwise separable convolution is proposed. Firstly, this network replaces the convolutional layer modules of the U-Net with redesigned depthwise separable convolution modules. Secondly, an modified Atrous spatial pyramid pooling (MASPP) is introduced to enlarge the receptive field and enhance the extraction of multiscale features. Thirdly, an attention gate (AG) structure is incorporated at the skip connections of the network to further enhance the segmentation accuracy of multiscale targets. Finally, Experimental evaluations are conducted using the ischemic stroke lesion segmentation 2022 challenge (ISLES2022) dataset. The proposed algorithm in this paper achieves Dice similarity coefficient (DSC), Hausdorff distance (HD), sensitivity (SEN), and precision (PRE) scores of 0.816 5, 3.668 1, 0.889 2, and 0.894 6, respectively, outperforming other mainstream segmentation algorithms. The experimental results demonstrate that the method in this paper effectively improves the segmentation of infarct lesions, and is expected to provide a reliable support for clinical diagnosis and treatment.

SFMnet: Edge detection of HABs based on spatial feature mapping encoder-decoder network

Achieving high-accuracy real-time recognition of marine red tide algae images is crucial in implementing red tide algae early warning systems. However, due to uneven lighting and the deposition of unknown suspended particles in seawater, real-time collected harmful algal blooms (HABs) microscopic images in marine engineering suffer from poor clarity and blurred cell contours, leading to lower accuracy of current edge detection models for microscopic images. Firstly, to enhance the quality of degraded images in the original HABs dataset, we design an image enhancement algorithm based on underwater image restoration models, which involves precise depth estimation of seawater scenes and fusion of transmission maps. This algorithm effectively eliminates chromatic aberration and enhances image edge details and contrast. Secondly, we propose a deep-learning model based on the VGG16 network. This model integrates multi-scale features through the Spatial Feature Mapping module to improve edge detection accuracy. Experimental results using real-time HABs microscopic image data show that the proposed enhancement method achieves the best results regarding contrast, image enhancement metrics, and clarity gradient. Moreover, the edge detection method achieves an ODS of 0.636 and an OIS of 0.701. The results demonstrate that performing edge detection after the image enhancement preprocess significantly improves detection accuracy.

Medical image super-resolution via transformer-based hierarchical encoder–decoder network

Medical image super-resolution via transformer-based hierarchical encoder–decoder network

MultiTumor Analyzer (MTA-20–55): A network for efficient classification of detected brain tumors from MRI images

Brain cancer, one of the leading causes of mortality worldwide, is caused by brain tumors. Early diagnosis of tumors and predicting their progression can help doctors to save lives. In this article, we have designed an automated approach for locating and classifying tumors from MRI images. The novelties of the research work include the following two stages: Developing an encoder-decoder type 20-Layered deep neural network (DNN) named MultiTumor Analyzer (MTA-20) with 15 down-sampling layers and 4 up-sampling layers, the segmentation is performed in the initial stage. Here, we have adhered a Leaky ReLU activation function instead of ReLU which learn a parameter with negative values that may have valuable information which is essential specifically for image segmentation. Further, a 55-layered DNN using multistage feature fusion is developed in the second stage of the work for the classification of localized tumors. The classification is performed using developed MultiTumor Analyzer (MTA-55) DNN with Softmax classifier. The efficacy of the designed network is validated using highly cited quantitative measures such as accuracy, sensitivity, specificity, dice similarity coefficient (DSC), precision, and F1-measure. It is observed that the proposed MTA-20 DNN attains the average accuracy, sensitivity, specificity, DSC, and precision of 99.2 %, 94.6 %, 99.3 %, 88 %, and 82.5 % respectively against seven state-of-the-art techniques. Also, it is found that, the proposed MTA-55 DNN provides the overall accuracy, recall, specificity, F1-measure, precision, and DSC of 99.8 %, 99.633 %, 99.844 %, 99.659 %, 99.689 %, and 99.656 % respectively as compared to thirteen state-of-the-art techniques. These results corroborate the superiority of the proposed technique.

Nighttime road scene image enhancement based on cycle-consistent generative adversarial network

During nighttime road scenes, images are often affected by contrast distortion, loss of detailed information, and a significant amount of noise. These factors can negatively impact the accuracy of segmentation and object detection in nighttime road scenes. A cycle-consistent generative adversarial network has been proposed to address this issue to improve the quality of nighttime road scene images. The network includes two generative networks with identical structures and two adversarial networks with identical structures. The generative network comprises an encoder network and a corresponding decoder network. A context feature extraction module is designed as the foundational element of the encoder-decoder network to capture more contextual semantic information with different receptive fields. A receptive field residual module is also designed to increase the receptive field in the encoder network.The illumination attention module is inserted between the encoder and decoder to transfer critical features extracted by the encoder to the decoder. The network also includes a multiscale discriminative network to discriminate better whether the image is a real high-quality or generated image. Additionally, an improved loss function is proposed to enhance the efficacy of image enhancement. Compared to state-of-the-art methods, the proposed approach achieves the highest performance in enhancing nighttime images, making them clearer and more natural.

High-quality AFM image acquisition of living cells by modified residual encoder-decoder network

Atomic force microscope enables ultra-precision imaging of living cells. However, atomic force microscope imaging is a complex and time-consuming process. The obtained images of living cells usually have low resolution and are easily influenced by noise leading to unsatisfactory imaging quality, obstructing the research and analysis based on cell images. Herein, an adaptive attention image reconstruction network based on residual encoder-decoder was proposed, through the combination of deep learning technology and atomic force microscope imaging supporting high-quality cell image acquisition. Compared with other learning-based methods, the proposed network showed higher peak signal-to-noise ratio, higher structural similarity and better image reconstruction performances. In addition, the cell images reconstructed by each method were used for cell recognition, and the cell images reconstructed by the proposed network had the highest cell recognition rate. The proposed network has brought insights into the atomic force microscope-based imaging of living cells and cell image reconstruction, which is of great significance in biological and medical research.

Uncertainty quantification for goal-oriented inverse problems via variational encoder-decoder networks

In this work, we describe a new approach that uses variational encoder-decoder (VED) networks for efficient uncertainty quantification for goal-oriented inverse problems. Contrary to standard inverse problems, these approaches are goal-oriented in that the goal is to estimate some quantities of interest (QoI) that are functions of the solution of an inverse problem, rather than the solution itself. Moreover, we are interested in computing uncertainty metrics associated with the QoI, thus utilizing a Bayesian approach for inverse problems that incorporates the prediction operator and techniques for exploring the posterior. This may be particularly challenging, especially for nonlinear, possibly unknown, operators and nonstandard prior assumptions. We harness recent advances in machine learning, i.e. VED networks, to describe a data-driven approach to large-scale inverse problems. This enables a real-time uncertainty quantification for the QoI. One of the advantages of our approach is that we avoid the need to solve challenging inversion problems by training a network to approximate the mapping from observations to QoI. Another main benefit is that we enable uncertainty quantification for the QoI by leveraging probability distributions in the latent and target spaces. This allows us to efficiently generate QoI samples and circumvent complicated or even unknown forward models and prediction operators. Numerical results from medical tomography reconstruction and nonlinear hydraulic tomography demonstrate the potential and broad applicability of the approach.

A multimodal multitask deep learning framework for vibrotactile feedback and sound rendering

Data-driven approaches are often utilized to model and generate vibrotactile feedback and sounds for rigid stylus-based interaction. Nevertheless, in prior research, these two modalities were typically addressed separately due to challenges related to synchronization and design complexity. To this end, we introduce a novel multimodal multitask deep learning framework. In this paper, we developed a comprehensive end-to-end data-driven system that encompasses the capture of contact acceleration signals and sound data from various texture surfaces. This framework introduces novel encoder-decoder networks for modeling and rendering vibrotactile feedback through an actuator while routing sound to headphones. The proposed encoder-decoder networks incorporate stacked transformers with convolutional layers to capture both local variability and overall trends within the data. To the best of our knowledge, this is the first attempt to apply transformer-based data-driven approach for modeling and rendering of vibrotactile signals as well as sounds during tool-surface interactions. In numerical evaluations, the proposed framework demonstrates a lower RMS error compared to state-of-the-art models for both vibrotactile signals and sound data. Additionally, subjective similarity evaluation also confirm the superiority of proposed method over state-of-the-art.

The fusion of lock-in phase images for the damage investigation in composites materials

Lock-in thermography (LIT) is a type of active thermography capable of detecting and evaluating the subsurface defects in composite materials. The phase or amplitude difference between the defect and sound regions can quantitatively determine the size and depth of defects. One limitation of LIT is that the optimal identification of defects located at specific depths is dispersed within phase images acquired at varying excitation frequencies. If the depth of the defects within the specimen is unknown, it is difficult to determine the excitation frequencies to achieve optimal contrast for all defects and display them in a single image. To address this challenge, a multi-frequency fused method based on lock-in thermography is proposed to improve the quality of defect detection. Initially, the optimal thermal wave excitation frequencies and the number of detection times are determined based on the theoretical solution. Subsequently, phase images at various excitation frequencies are extracted and enhanced using a specified scheme. Finally, we develop an unsupervised encoder-decoder network that combines dense connections and residual modules to fuse these phase images into a single image containing defects at various depths. An experiment is conducted to detect carbon fiber reinforced polymer (CFRP) laminate containing defects with different depths and sizes. The results demonstrate that the proposed method can broaden the detection range of defect depths and improve the quality of the inspected images, providing superior performance compared to traditional sequential image processing methods.

Autonomous Image-Based Corrosion Detection in Steel Structures Using Deep Learning.

Steel structures are susceptible to corrosion due to their exposure to the environment. Currently used non-destructive techniques require inspector involvement. Inaccessibility of the defective part may lead to unnoticed corrosion, allowing the corrosion to propagate and cause catastrophic structural failure over time. Autonomous corrosion detection is essential for mitigating these problems. This study investigated the effect of the type of encoder-decoder neural network and the training strategy that works the best to automate the segmentation of corroded pixels in visual images. Models using pre-trained DesnseNet121 and EfficientNetB7 backbones yielded 96.78% and 98.5% average pixel-level accuracy, respectively. Deeper EffiecientNetB7 performed the worst, with only 33% true-positive values, which was 58% less than ResNet34 and the original UNet. ResNet 34 successfully classified the corroded pixels, with 2.98% false positives, whereas the original UNet predicted 8.24% of the non-corroded pixels as corroded when tested on a specific set of images exclusive to the investigated training dataset. Deep networks were found to be better for transfer learning than full training, and a smaller dataset could be one of the reasons for performance degradation. Both fully trained conventional UNet and ResNet34 models were tested on some external images of different steel structures with different colors and types of corrosion, with the ResNet 34 backbone outperforming conventional UNet.

Applications of encoding-decoding network on numerical simulation mapping for propeller wake

Numerical Simulation methods of RANS and LES have been widely used to simulate the velocity field induced by propellers. In order to combine advantages of quick computation time from RANS and detailed information from LES, this paper innovatively proposed a customized data-driven encoding-decoding network to achieve mapping from RANS results to LES results. Firstly, an encoding-decoding neural network model was constructed to extract features from RANS/LES for propeller wake. The extracted features with additional information concatenation during the encoding steps were then used to reconstruct the original samples and got a reconstruction error of 1%, which proved the effectiveness of feature extraction ability of the proposed model. The proposed model was then used to map directly from RANS to LES data with a better performance compared to traditional PCA feature extraction and mapping method. Afterwards, the proposed network was applied to two more RANS to LES mapping scenarios to validate the generalization ability. The model could map a coarse-grid RANS to LES of ship propeller with less than 9% MSE error. Similarly, mapped LES of nominal wake was 98% more alike the original LES samples than the RANS samples. As a result, the proposed model could be used either to extract features for potential feature-based performance prediction, or as a quicker method for detailed simulation for propeller wake and nominal wake, thus expedites the design process of ship design.

Highly efficient encoder-decoder network based on multi-scale edge enhancement and dilated convolution for LDCT image denoising

Highly efficient encoder-decoder network based on multi-scale edge enhancement and dilated convolution for LDCT image denoising

MISP-Fuse: A progressive fusion network guided by Multi-Information supervision

The fusion of infrared and visible images aims to synthesize an image that combines the advantageous characteristics of both image sources. Nevertheless, existing image fusion algorithms often struggle to extract relevant feature information from the source images and find it challenging to balance the significance of different modality information within the source images. This leads to suboptimal fusion results and inadequately serve advanced downstream visual tasks. In order to confront this challenge, a novel image fusion algorithm, denoted as Multiple Information Supervised Progressive Fusion Network (MISP-Fuse), has been proposed. Specifically, MISP-Fuse employs an innovative and robust multi-stage encoder-decoder network called Full Scale Feature Residual network (FSFR) to extract spatial context and detailed feature information corresponding to infrared and visible images. Within this encoder-decoder network, source image feature information is progressively extracted at different stages, which are recombined at multiple scales and distilled for crucial information. Finally, the fusion image is generated through a spatial localization module named as Spatial Localization Network (SLNet). MISP-Fuse incorporates a multi-information supervision mechanism to establish a linkage between different modality information in the infrared and visible source images. It ensures that the resulting fused image not only aligns with human visual perception but also effectively serves advanced downstream visual tasks. The comparative experiments utilizing diverse image fusion benchmark datasets has been conducted. In comparison to other algorithms, MISP-Fuse demonstrated significant enhancements in comprehensive image fusion metrics, including Average Gradient (AG), Sum of Correlated Differences (SCD), and Correlation Coefficient (CC).

Predicting 360° Video Saliency: A ConvLSTM Encoder-Decoder Network With Spatio-Temporal Consistency

Predicting 360° Video Saliency: A ConvLSTM Encoder-Decoder Network With Spatio-Temporal Consistency

Enhancing particle string detection in electrorheological plasmas using asymmetrical kernel convolutional networks

Under different plasma conditions and electric fields in a complex plasma the plasma particles organize themselves in a string-like or chain-like manner. A phase transition from string-like to an isotropic particle distribution is observed at different electrical conditions. The streaming of charged ions around plasma particles with the surrounding electric field gives the plasma its electrorheological properties. The visibility of individual particles in a complex plasma opens up the opportunity to examine properties and phase transitions of such electrorheological fluids in detail. Because of the limited one-dimensional symmetry, determining the configuration of a particle and recognizing strings in particle distributions is not always straightforward. Several approaches have already been used to analyse particle clouds while either considering each particle locally or considering the particle cloud as a whole without providing information about single particle configurations. This paper presents a new machine learning approach that takes advantage of particle distributions over the entire particle cloud and detects all string-like particles at once, using a convolutional neural network in form of an encoder-decoder network with asymmetric kernel convolutions. This not only enhances the result quality but also accelerates the evaluation process, possibly enabling real-time analyses on electrorheological phase transitions, while achieving an accuracy of over 95% on manually labelled data.

Crop pest identification using deep network based extracted features and MobileENet in smart agriculture

AbstractAgriculture has been considered an important source of food for humans throughout history. Plant pests cause significant damage to crops and reduce the productivity of global crop yields. Therefore, it is important to identify the plant pest at an earlier stage in order to minimize crop losses and use pesticides optimally. This paper develops the MobileENet deep learning architecture for accurate plant pest identification with less computational effort. The input images are pre‐processed, and the features are extracted using a deep convolutional encoder–decoder network (DCEDN). The proposed classification approach solves the problems of over‐fitting regularization, batch normalization, and dropout layers. Due to the minimum computing size and factorization process, the classification performance is increased. It extracts discriminatory feature information by eliminating redundant background information. The performance of the proposed approach is evaluated on the IP102 dataset, and the performance is compared with existing deep learning‐based approaches. The performance metrics, such as accuracy, precision, recall, and so forth, are considered to evaluate the performance of the proposed plant pest identification approach. The accuracy performance of the proposed approach is improved to 98.83% with less information loss.

A novel fusion of genetic grey wolf optimization and kernel extreme learning machines for precise diabetic eye disease classification.

In response to the growing number of diabetes cases worldwide, Our study addresses the escalating issue of diabetic eye disease (DED), a significant contributor to vision loss globally, through a pioneering approach. We propose a novel integration of a Genetic Grey Wolf Optimization (G-GWO) algorithm with a Fully Convolutional Encoder-Decoder Network (FCEDN), further enhanced by a Kernel Extreme Learning Machine (KELM) for refined image segmentation and disease classification. This innovative combination leverages the genetic algorithm and grey wolf optimization to boost the FCEDN's efficiency, enabling precise detection of DED stages and differentiation among disease types. Tested across diverse datasets, including IDRiD, DR-HAGIS, and ODIR, our model showcased superior performance, achieving classification accuracies between 98.5% to 98.8%, surpassing existing methods. This advancement sets a new standard in DED detection and offers significant potential for automating fundus image analysis, reducing reliance on manual examination, and improving patient care efficiency. Our findings are crucial to enhancing diagnostic accuracy and patient outcomes in DED management.