Decoder Module Research Articles

Anti-Aliasing Attention U-net Model for Skin Lesion Segmentation

The need for a lightweight and reliable segmentation algorithm is critical in various biomedical image-prediction applications. However, the limited quantity of data presents a significant challenge for image segmentation. Additionally, low image quality negatively impacts the efficiency of segmentation, and previous deep learning models for image segmentation require large parameters with hundreds of millions of computations, resulting in high costs and processing times. In this study, we introduce a new lightweight segmentation model, the mobile anti-aliasing attention u-net model (MAAU), which features both encoder and decoder paths. The encoder incorporates an anti-aliasing layer and convolutional blocks to reduce the spatial resolution of input images while avoiding shift equivariance. The decoder uses an attention block and decoder module to capture prominent features in each channel. To address data-related problems, we implemented data augmentation methods such as flip, rotation, shear, translate, and color distortions, which enhanced segmentation efficiency in the international Skin Image Collaboration (ISIC) 2018 and PH2 datasets. Our experimental results demonstrated that our approach had fewer parameters, only 4.2 million, while it outperformed various state-of-the-art segmentation methods.

Controllable Image Captioning with Feature Refinement and Multilayer Fusion

Image captioning is the task of automatically generating a description of an image. Traditional image captioning models tend to generate a sentence describing the most conspicuous objects, but fail to describe a desired region or object as human. In order to generate sentences based on a given target, understanding the relationships between particular objects and describing them accurately is central to this task. In detail, information-augmented embedding is used to add prior information to each object, and a new Multi-Relational Weighted Graph Convolutional Network (MR-WGCN) is designed for fusing the information of adjacent objects. Then, a dynamic attention decoder module selectively focuses on particular objects or semantic contents. Finally, the model is optimized by similarity loss. The experiment on MSCOCO Entities demonstrates that IANR obtains, to date, the best published CIDEr performance of 124.52% on the Karpathy test split. Extensive experiments and ablations on both the MSCOCO Entities and the Flickr30k Entities demonstrate the effectiveness of each module. Meanwhile, IANR achieves better accuracy and controllability than the state-of-the-art models under the widely used evaluation metric.

Intelligent autoencoder for space-time-coding digital metasurfaces

Space-time-coding (STC) digital metasurfaces provide a powerful platform for simultaneous spatiotemporal modulations of electromagnetic waves. Therefore, the fast and accurate generation of STC matrices based on desired harmonic scattering patterns can help STC metasurfaces enhance their practicality in various applications. Here, we propose a physics-driven vector-quantized (PD-VQ) intelligent autoencoder model that consists of an encoder, a vector-quantizer layer, and a physics-driven decoder. The physical operation mechanism between the STC matrix and the harmonic scattering pattern is introduced into the decoding module of the PD-VQ intelligent autoencoder, so that the autoencoder can be trained in an unsupervised manner without the need for large amount of manually labeled data. Taking a target harmonic scattering pattern as input, the trained PD-VQ autoencoder can quickly output the optimized discrete STC matrix, which takes only about 78 ms. We present a series of simulation examples to verify the reliability and accuracy of the proposed approach and also demonstrate its good generalization capability. Based on the proposed PD-VQ intelligent autoencoder, the STC digital metasurfaces enable agile multi-frequency harmonic beamforming.

A novel encoder-decoder model based on Autoformer for air quality index prediction.

Rapid economic development has led to increasingly serious air quality problems. Accurate air quality prediction can provide technical support for air pollution prevention and treatment. In this paper, we proposed a novel encoder-decoder model named as Enhanced Autoformer (EnAutoformer) to improve the air quality index (AQI) prediction. In this model, (a) The enhanced cross-correlation (ECC) is proposed for extracting the temporal dependencies in AQI time series; (b) Combining the ECC with the cross-stage feature fusion mechanism of CSPDenseNet, the core module CSP_ECC is proposed for improving the computational efficiency of the EnAutoformer. (c) The time series decomposition and dilated causal convolution added in the decoder module are exploited to extract the finer-grained features from the original AQI data and improve the performance of the proposed model for long-term prediction. The real-world air quality datasets collected from Lanzhou are used to validate the performance of our prediction model. The experimental results show that our EnAutoformer model can greatly improve the prediction accuracy compared to the baselines and can be used as a promising alternative for complex air quality prediction.

Trimap-guided feature mining and fusion network for natural image matting

Mining interesting objects with trimap guidance and fusing multi-level features are two important issues for trimap-based matting. For feature mining, existing methods simply feed trimaps to an encoder network and mine features evenly in different regions with a decoding module, which lacks an efficient and effective way to mine interesting objects pointed out by trimaps. For emerging content-based feature fusion, most existing matting methods only focus on local features which lack the guidance of a global feature with strong semantic information related to the interesting object. In this paper, we propose a trimap-guided feature mining and fusion network consisting of our trimap-guided non-background multi-scale pooling (TMP) module and global-local context-aware fusion (GLF) modules. Considering that trimap provides strong semantic guidance, our TMP module focuses effective feature mining on interesting objects under the guidance of trimap without extra parameters. Furthermore, our GLF modules use global semantic information of interesting objects mined by our TMP module to guide an effective global-local context-aware multi-level feature fusion. In addition, we build a common interesting object matting (CIOM) dataset to advance high-quality image matting. Experimental results on the Composition-1k test set, Alphamatting benchmark, and our CIOM test set demonstrate that our method outperforms state-of-the-art approaches. Codes and models will be publicly available at https://github.com/Serge-weihao/TMF-Matting.

Graph Autoencoder with Preserving Node Attribute Similarity.

The graph autoencoder (GAE) is a powerful graph representation learning tool in an unsupervised learning manner for graph data. However, most existing GAE-based methods typically focus on preserving the graph topological structure by reconstructing the adjacency matrix while ignoring the preservation of the attribute information of nodes. Thus, the node attributes cannot be fully learned and the ability of the GAE to learn higher-quality representations is weakened. To address the issue, this paper proposes a novel GAE model that preserves node attribute similarity. The structural graph and the attribute neighbor graph, which is constructed based on the attribute similarity between nodes, are integrated as the encoder input using an effective fusion strategy. In the encoder, the attributes of the nodes can be aggregated both in their structural neighborhood and by their attribute similarity in their attribute neighborhood. This allows performing the fusion of the structural and node attribute information in the node representation by sharing the same encoder. In the decoder module, the adjacency matrix and the attribute similarity matrix of the nodes are reconstructed using dual decoders. The cross-entropy loss of the reconstructed adjacency matrix and the mean-squared error loss of the reconstructed node attribute similarity matrix are used to update the model parameters and ensure that the node representation preserves the original structural and node attribute similarity information. Extensive experiments on three citation networks show that the proposed method outperforms state-of-the-art algorithms in link prediction and node clustering tasks.

Defect transformer: An efficient hybrid transformer architecture for surface defect detection

Surface defect detection is an extremely crucial step to ensure the quality of industrial products. Nowadays, convolutional neural networks (CNNs) based on encoder–decoder architecture have achieved tremendous success in various defect detection tasks. However, the intrinsic locality of convolution prevents them from modeling long-range interactions explicitly, making it difficult to distinguish pseudo-defects in cluttered backgrounds. Recent transformers are especially skilled at learning global image dependencies, but with limited local structural information for the refined defect location. To overcome the above limitations, we incorporate CNN and transformer into an efficient hybrid transformer architecture for defect detection, termed Defect Transformer (DefT), to capture local and non-local relationships collaboratively. Specifically, in the encoder module, a convolutional stem block is firstly adopted to retain more spatial details. Then, the patch aggregation blocks are used to generate multi-scale representation with four hierarchies, each of them is followed by a series of DefT blocks, which respectively include a locally position-aware block for local position encoding, a lightweight multi-pooling self-attention to model multi-scale global contextual relationships with good computational efficiency, and a convolutional feed-forward network for feature transformation and further local information learning. Finally, a simple but effective decoder module is constructed to gradually recover spatial details from the skip connections in the encoder. Extensive experiments on three datasets demonstrate the superiority and efficiency of our method compared with other deeper and complex CNN- and transformer-based networks.

MCANet: A Multi-Branch Network for Cloud/Snow Segmentation in High-Resolution Remote Sensing Images

Because clouds and snow block the underlying surface and interfere with the information extracted from an image, the accurate segmentation of cloud/snow regions is essential for imagery preprocessing for remote sensing. Nearly all remote sensing images have a high resolution and contain complex and diverse content, which makes the task of cloud/snow segmentation more difficult. A multi-branch convolutional attention network (MCANet) is suggested in this study. A double-branch structure is adopted, and the spatial information and semantic information in the image are extracted. In this way, the model’s feature extraction ability is improved. Then, a fusion module is suggested to correctly fuse the feature information gathered from several branches. Finally, to address the issue of information loss in the upsampling process, a new decoder module is constructed by combining convolution with a transformer to enhance the recovery ability of image information; meanwhile, the segmentation boundary is repaired to refine the edge information. This paper conducts experiments on the high-resolution remote sensing image cloud/snow detection dataset (CSWV), and conducts generalization experiments on two publicly available datasets (HRC_WHU and L8 SPARCS), and the self-built cloud and cloud shadow dataset. The MIOU scores on the four datasets are 92.736%, 91.649%, 80.253%, and 94.894%, respectively. The experimental findings demonstrate that whether it is for cloud/snow detection or more complex multi-category detection tasks, the network proposed in this paper can completely restore the target details, and it provides a stronger degree of robustness and superior segmentation capabilities.

Time Domain Analysis of NB-IoT Signals

The NB-IoT (NarrowBand-Internet of Things) radio technology is now widely implemented by mobile phone network operators to support the communication of IoT devices such as smart meters, insurance black boxes for cars, network connected waste bins, smart bicycles. In the present work, some LTE800 cells of different mobile phone network operators implementing the NB-IoT technology in the guard band mode have been investigated. The signals, consisting of a PRB (Physical Resource Block) 180 kHz wide, have been analyzed and characterized in the time domain by means of a narrow band instrumental chain equipped with a Rohde & Schwarz FSH8 spectrum analyzer. Time domain analysis allows us to identify, within the transmission frame, the position of the NB-IoT signaling channels such as the Narrowband Reference Signal (NRS), the primary (NPSS) and secondary (NSSS) synchronization signals and the broadcast channel (NPBCH), but, above all, to measure the power received during the transmission of the NRS. This value has been compared with that measured by the NB-IoT decoding module supplied on the same analyzer, in order to verify the equivalence of these measurement methods. This would allow use of a more diffuse and cheaper instrumentation rather than more expensive vector analyzers, currently required to assess electric fields due to the NB-IoT signals through the extrapolation techniques set by Italian CEI 211-7/E technical standard.

Research on reconstruction algorithm of power quality data based on self-encoding network

Taking on the problems of unstable reconstruction performance and high reconstruction method computational cost induced by the unpredictability of the measurement matrix during power quality signal reconstruction, this research provides a power quality reconstruction model based on a self-encoding network and compressed sensing for the first time. The model includes a noise-adding module, an encoder module and a decoder module. The noise-adding module adds a specific amount of white noise in relation to the original signal to simulate the disturbance signal collected in the real scene. The encoder module uses a nonlinear measurement method to observe the power quality signal, and the decoder module completes the signal through a compressed sampling matching tracking algorithm. Refactoring. The outcomes of the experiment reveal that the compressed reconstruction model proposed in this paper significantly improves the efficiency and stability of power quality signal reconstruction.

EvaGoNet: An integrated network of variational autoencoder and Wasserstein generative adversarial network with gradient penalty for binary classification tasks

Feature engineering is an effective method for solving classification problems. Many existing feature engineering studies have focused on image or video data and not on structured data. This study proposes EvaGoNet, which refines the decoder module of the Gaussian mixture variational autoencoder using the Wasserstein generative adversarial network with gradient penalty (WGANgp) and embeds the top-ranked original features to update the latent features based on their discriminative powers. Comprehensive experiments show that EvaGoNet-encoded features outperform existing classifiers on 12 benchmark datasets, particularly on the small, imbalanced datasets col (accuracy = 0.8581), spe (accuracy = 1.0000), and leu (accuracy = 0.8021). EvaGoNet-engineered features improve binary classification task outcomes on six high-dimensional, imbalanced bioOMIC datasets. EvaGoNet achieves a medium-ranked training speed among the compared algorithms and considerably fast prediction speeds in the predictions of the testing samples. Therefore, EvaGoNet can be a candidate feature engineering framework for many practical applications that require one training procedure and many prediction tasks of the testing samples. EvaGoNet is implemented in Python TensorFlow and is available at https://healthinformaticslab.org/supp/resources.php.

Intelligent Monitoring System of Access Equipment in Dispatching Exchange Network Based on Multi-layer Immune Model

To visualize the soft switch traffic and traffic flow of the whole network scheduling, it is necessary to monitor the scheduling of the switching network intelligently. The original system uses fixed-point mode to monitor devices, which can not anchor multiple groups of access devices simultaneously, resulting in errors in monitoring results. In the hardware design of the system, a multilayer immune model is used to establish a three-level visualization partition, and a monitoring system framework for access devices is constructed. In the software design, the detection module, the prompt module, and the automatic decoding module are set up, respectively, and the monitoring process of the dispatching switching network is configured. The access equipment of the dispatching exchange network is intelligently monitored and controlled by an immune model with a three-layer structure. The experimental results show that the data obtained by this system is consistent with the real data and has an application effect by taking the communication data of power dispatching in a provincial power system as the test object and monitoring the communication data of multiple links in a year.

DCENet: A dynamic correlation evolve network for short-term traffic prediction

Graph neural networks (GNNs) have been extensively employed in traffic prediction tasks due to their excellent capturing capabilities of spatial dependence. However, the majority of GNN-based approaches tend to employ static graphs, whereas they evolve over time and vary dynamics in real-world traffic situations. It is challenging to capture the dynamic spatial–temporal evolution characteristics of traffic data. To address this problem, we propose a dynamic correlation evolve network (DCENet) for short-term traffic prediction. To be specific, we develop a dynamic correlation self-attention (DCSA) module, which captures dynamic node associations adaptively. In this way, the model acquires new node embedding features without explicitly constructing a new graph structure. Then, an evolution encoder–decoder (EED) module is built to learn the interactions of dynamic features and output future traffic states. The experiments are conducted on two real-world datasets, and the results show that the DCENet outperformers baseline models for most of the cases.

Encoder-decoder network with RMP for tongue segmentation.

Tongue and its movements can be used for several medical-related tasks, such as identifying a disease and tracking a rehabilitation. To be able to focus on a tongue region, the tongue segmentation is needed to compute a region of interest for a further analysis. This paper proposes an encoder-decoder CNN-based architecture for segmenting a tongue in an image. The encoder module is mainly used for the tongue feature extraction, while the decoder module is used to reconstruct a segmented tongue from the extracted features based on training images. In addition, the residual multi-kernel pooling (RMP) is also applied into the proposed network to help in encoding multiple scales of the features. The proposed method is evaluated on two publicly available datasets under a scenario of front view and one tongue posture. It is then tested on a newly collected dataset of five tongue postures. The reported performances show that the proposed method outperforms existing methods in the literature. In addition, the re-training process could improve applying the trained model on unseen dataset, which would be a necessary step of applying the trained model on the real-world scenario.

Dual encoder–decoder-based deep polyp segmentation network for colonoscopy images

Detection of colorectal polyps through colonoscopy is an essential practice in prevention of colorectal cancers. However, the method itself is labor intensive and is subject to human error. With the advent of deep learning-based methodologies, and specifically convolutional neural networks, an opportunity to improve upon the prognosis of potential patients suffering with colorectal cancer has appeared with automated detection and segmentation of polyps. Polyp segmentation is subject to a number of problems such as model overfitting and generalization, poor definition of boundary pixels, as well as the model’s ability to capture the practical range in textures, sizes, and colors. In an effort to address these challenges, we propose a dual encoder–decoder solution named Polyp Segmentation Network (PSNet). Both the dual encoder and decoder were developed by the comprehensive combination of a variety of deep learning modules, including the PS encoder, transformer encoder, PS decoder, enhanced dilated transformer decoder, partial decoder, and merge module. PSNet outperforms state-of-the-art results through an extensive comparative study against 5 existing polyp datasets with respect to both mDice and mIoU at 0.863 and 0.797, respectively. With our new modified polyp dataset we obtain an mDice and mIoU of 0.941 and 0.897 respectively.

Method of Building Detection in Optical Remote Sensing Images Based on SegFormer.

An appropriate detection network is required to extract building information in remote sensing images and to relieve the issue of poor detection effects resulting from the deficiency of detailed features. Firstly, we embed a transposed convolution sampling module fusing multiple normalization activation layers in the decoder based on the SegFormer network. This step alleviates the issue of missing feature semantics by adding holes and fillings, cascading multiple normalizations and activation layers to hold back over-fitting regularization expression and guarantee steady feature parameter classification. Secondly, the atrous spatial pyramid pooling decoding module is fused to explore multi-scale contextual information and to overcome issues such as the loss of detailed information on local buildings and the lack of long-distance information. Ablation experiments and comparison experiments are performed on the remote sensing image AISD, MBD, and WHU dataset. The robustness and validity of the improved mechanism are demonstrated by control groups of ablation experiments. In comparative experiments with the HRnet, PSPNet, U-Net, DeepLabv3+ networks, and the original detection algorithm, the mIoU of the AISD, the MBD, and the WHU dataset is enhanced by 17.68%, 30.44%, and 15.26%, respectively. The results of the experiments show that the method of this paper is superior to comparative methods such as U-Net. Furthermore, it is better for integrity detection of building edges and reduces the number of missing and false detections.

Toward attention-based learning to predict the risk of brain degeneration with multimodal medical data.

Brain degeneration is commonly caused by some chronic diseases, such as Alzheimer's disease (AD) and diabetes mellitus (DM). The risk prediction of brain degeneration aims to forecast the situation of disease progression of patients in the near future based on their historical health records. It is beneficial for patients to make an accurate clinical diagnosis and early prevention of disease. Current risk predictions of brain degeneration mainly rely on single-modality medical data, such as Electronic Health Records (EHR) or magnetic resonance imaging (MRI). However, only leveraging EHR or MRI data for the pertinent and accurate prediction is insufficient because of single-modality information (e.g., pixel or volume information of image data or clinical context information of non-image data). Several deep learning-based methods have used multimodal data to predict the risks of specified diseases. However, most of them simply integrate different modalities in an early, intermediate, or late fusion structure and do not care about the intra-modal and intermodal dependencies. A lack of these dependencies would lead to sub-optimal prediction performance. Thus, we propose an encoder-decoder framework for better risk prediction of brain degeneration by using MRI and EHR. An encoder module is one of the key components and mainly focuses on feature extraction of input data. Specifically, we introduce an encoder module, which integrates intra-modal and inter-modal dependencies with the spatial-temporal attention and cross-attention mechanism. The corresponding decoder module is another key component and mainly parses the features from the encoder. In the decoder module, a disease-oriented module is used to extract the most relevant disease representation features. We take advantage of a multi-head attention module followed by a fully connected layer to produce the predicted results. As different types of AD and DM influence the nature and severity of brain degeneration, we evaluate the proposed method for three-class prediction of AD and three-class prediction of DM. Our results show that the proposed method with integrated MRI and EHR data achieves an accuracy of 0.859 and 0.899 for the risk prediction of AD and DM, respectively. The prediction performance is significantly better than the benchmarks, including MRI-only, EHR-only, and state-of-the-art multimodal fusion methods.

Dual Residual Denoising Autoencoder with Channel Attention Mechanism for Modulation of Signals

Aiming to address the problems of the high bit error rate (BER) of demodulation or low classification accuracy of modulation signals with a low signal-to-noise ratio (SNR), we propose a double-residual denoising autoencoder method with a channel attention mechanism, referred to as DRdA-CA, to improve the SNR of modulation signals. The proposed DRdA-CA consists of an encoding module and a decoding module. A squeeze-and-excitation (SE) ResNet module containing one residual connection is modified and then introduced into the autoencoder as the channel attention mechanism, to better extract the characteristics of the modulation signals and reduce the computational complexity of the model. Moreover, the other residual connection is further added inside the encoding and decoding modules to optimize the network degradation problem, which is beneficial for fully exploiting the multi-level features of modulation signals and improving the reconstruction quality of the signal. The ablation experiments prove that both the improved SE module and dual residual connections in the proposed method play an important role in improving the denoising performance. The subsequent experimental results show that the proposed DRdA-CA significantly improves the SNR values of eight modulation types in the range of -12 dB to 8 dB. Especially for 16QAM and 64QAM, the SNR is improved by 8.38 dB and 8.27 dB on average, respectively. Compared to the DnCNN denoising method, the proposed DRdA-CA makes the average classification accuracy increase by 67.59∼74.94% over the entire SNR range. When it comes to the demodulation, compared with the RLS and the DnCNN denoising algorithms, the proposed denoising method reduces the BER of 16QAM by an average of 63.5% and 40.5%, and reduces the BER of 64QAM by an average of 46.7% and 18.6%. The above results show that the proposed DRdA-CA achieves the optimal noise reduction effect.

SABV-Depth: A biologically inspired deep learning network for monocular depth estimation

Monocular depth estimation makes it possible for machines to perceive the real world. The prediction performance of the depth estimation network based on deep learning will be affected due to the depth of the deep network and the locality of convolution operations. The imitation of the biological visual system and its functional structure is becoming a research hotspot. In this paper, we study the interpretability relationship between the biological visual system and the monocular depth estimation network. By concretizing the attention mechanism in biological vision, we propose a monocular depth estimation network based on the self-attention mechanism, named SABV-Depth, which can improve prediction accuracy. Inspired by the biological visual interaction mechanism, we focus on the information transfer between each module of the network and improve the information retention ability, and enable the network to output a depth map with rich object information and detailed information. Further, a decoder module with an inner-connection is proposed to recover depth maps with sharp edge contours. Our method is experimentally validated on the KITTI dataset and NYU Depth V2 dataset. The results show that compared with other works, the proposed method improves prediction accuracy. Meanwhile, the depth map has more object information and detail information, and a better edge information processing effect.

MCAFNet: A Multiscale Channel Attention Fusion Network for Semantic Segmentation of Remote Sensing Images

Semantic segmentation for urban remote sensing images is one of the most-crucial tasks in the field of remote sensing. Remote sensing images contain rich information on ground objects, such as shape, location, and boundary and can be found in high-resolution remote sensing images. It is exceedingly challenging to identify remote sensing images because of the large intraclass variance and low interclass variance caused by these objects. In this article, we propose a multiscale hierarchical channel attention fusion network model based on a transformer and CNN, which we name the multiscale channel attention fusion network (MCAFNet). MCAFNet uses ResNet-50 and Vit-B/16 to learn the global–local context, and this strengthens the semantic feature representation. Specifically, a global–local transformer block (GLTB) is deployed in the encoder stage. This design handles image details at low resolution and extracts global image features better than previous methods. In the decoder module, a channel attention optimization module and a fusion module are added to better integrate high- and low-dimensional feature maps, which enhances the network’s ability to obtain small-scale semantic information. The proposed method is conducted on the ISPRS Vaihingen and Potsdam datasets. Both quantitative and qualitative evaluations show the competitive performance of MCAFNet in comparison to the performance of the mainstream methods. In addition, we performed extensive ablation experiments on the Vaihingen dataset in order to test the effectiveness of multiple network components.