Convolutional Encoder Research Articles

A new model based on multi-aspect images and complex-valued neural network for synthetic aperture radar automatic target recognition

ABSTRACT During the last decades, Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) has been vastly utilized in the military services and civil studies. Because of the sensibility of the SAR images of the imaging in the azimuth dimension, using the multi-aspect SAR image sequence is more practical than the single-aspect SAR image to achieve the superior classification accuracy in an actual SAR ATR task. At present time, multi-aspect SAR ATR models mostly utilize Recurrent Neural Networks (RNN) that depend on the sequence between samples and so suffer from the lack of information. In a practical work, a huge amount of training set is needed to train a deep learning network accurately, but it is so costly work to extract multi-aspect SAR images. So, in this article, a new model of multi-aspect SAR ATR is proposed by using a self-attention model. The proposed self-attention model is utilized to calculate the internal correlation between the original SAR images. At the same time, to develop the anti-noise capability of the proposed model and decrease the dependency on a huge amount of training data, a Convolutional Auto Encoder (CAE) is designed and utilized in the feature extraction section of the proposed model. On the other hand, unlike existing traditional methods, in this work we use both amplitude and phase information of SAR images to devolve the training process of the proposed model. It should be noted that all of the parameters of the proposed network are developed to a complex domain. In addition a complex backpropagation algorithm by using the gradient based model is used for training the network. In the end, experiments is obtained by using two MSTAR data set (the MSTAR-SOC and EOC). The experimental results prove that the proposed model not only can achieve a high recognitions rate on the case of sufficient training sample but also can obtain an acceptable rate in the case of small training samples in different complex situations. In addition, the simulation results demonstrate that by using the encoder of CAE in the proposed model, the whole configuration of the proposed model achieves the anti-noise capability that is a valuable benefit of any practical SAR ATR task.

Self‐supervised non‐rigid structure from motion with improved training of Wasserstein GANs

AbstractThis study proposes a self‐supervised method to reconstruct 3D limbic structures from 2D landmarks extracted from a single view. The loss of self‐consistency can be reduced by performing a random orthogonal projection of the reconstructed 3D structure. Thus, the training process can be self‐supervised by using geometric self‐consistency in the reconstruction–projection–reconstruction process. The self‐supervised network mainly consists of graph convolution and Transformer encoders. This network is called the SS‐Graphformer. By adding a discriminator, the SS‐Graphformer is used as a generator to form a Wasserstein Generative Adversarial Network architecture with a Gradient Penalty to improve the accuracy of the reconstruction. It is experimentally demonstrated that the addition of the 2D structure discriminator can significantly improve the accuracy of the reconstruction.

Real-Time Deep-Learned Reconstruction for a Scanning Intraoperative Probe

Accurate delineation of the boundary between cancerous and healthy tissue during cancer resection surgeries is important to ensure complete removal of cancerous cells while preserving healthy tissue. Labeling cancer cells with radiotracers, and then using a probe during surgery to detect the radiotracer distribution, is a potential solution for accurate tumor localization and hence better surgical outcomes. This work explores the feasibility of using deep learning to reconstruct a radiotracer distribution from data acquired by an intraoperative probe. The probe’s sensor array outputs (SAOs), obtained by scanning the probe over a region of interest, are supplied to the deep network, which then outputs a reconstructed radiotracer distribution for the region of interest. This initial work demonstrates that the deep network used here, a convolutional encoder–decoder (CED), can successfully reconstruct simulated 2-D radiotracer distributions from synthesized input data. However, the network was unable to generalize reliably when tested with count levels not present in the training set. Therefore, the network must be trained with desired count levels or else should include estimation of epistemic uncertainty to avoid misleading outcomes. We also show that test-time augmentation can improve reconstructed image quality, and hence can also be used to reduce the amount of training data required.

An Arbitrary Scale Super-Resolution Approach for 3D MR Images via Implicit Neural Representation

High Resolution (HR) medical images provide rich anatomical structure details to facilitate early and accurate diagnosis. In magnetic resonance imaging (MRI), restricted by hardware capacity, scan time, and patient cooperation ability, isotropic 3-dimensional (3D) HR image acquisition typically requests long scan time and, results in small spatial coverage and low signal-to-noise ratio (SNR). Recent studies showed that, with deep convolutional neural networks, isotropic HR MR images could be recovered from low-resolution (LR) input via single image super-resolution (SISR) algorithms. However, most existing SISR methods tend to approach scale-specific projection between LR and HR images, thus these methods can only deal with fixed up-sampling rates. In this paper, we propose ArSSR, an Arbitrary Scale Super-Resolution approach for recovering 3D HR MR images. In the ArSSR model, the LR image and the HR image are represented using the same implicit neural voxel function with different sampling rates. Due to the continuity of the learned implicit function, a single ArSSR model is able to achieve arbitrary and infinite up-sampling rate reconstructions of HR images from any input LR image. Then the SR task is converted to approach the implicit voxel function via deep neural networks from a set of paired HR and LR training examples. The ArSSR model consists of an encoder network and a decoder network. Specifically, the convolutional encoder network is to extract feature maps from the LR input images and the fully-connected decoder network is to approximate the implicit voxel function. Experimental results on three datasets show that the ArSSR model can achieve state-of-the-art SR performance for 3D HR MR image reconstruction while using a single trained model to achieve arbitrary up-sampling scales.

RPConvformer: A novel Transformer-based deep neural networks for traffic flow prediction

Traffic prediction problem is one of the essential tasks of intelligent transportation system (ITS), alleviating traffic congestion effectively and promoting the intelligent development of urban traffic. To accommodate long-range dependencies, Transformer-based methods have been used in traffic prediction tasks due to the parallelizable processing of sequences and explanation of attention matrices compared with recurrent neural units (RNNs). However, the Transformer-based model has two limitations, on the one hand, it ignores the local correlation in the traffic state in its parallel processing of the sequence, on the other hand, the absolute positional embedding is adopted to represent the positional relationship of time nodes is destroyed when it comes to calculate attention score. To address two embarrassing shortcomings, a novel framework called RPConvformer is proposed, where the improved parts are 1D causal convolutional sequence embedding and relative position encoding. In sequence embedding, we develop a sequence embedding layer composed of convolutional units, which consist of origin 1D convolutional and 1D causal convolutional. The size of the receptive field of the convolution can focus on the local region correlation of the sequence. In relative position encoding, we introduce a bias vector to automatically learn the relative position information of time nodes when linearly mapping the feature tensor. We respect the encoding and decoding framework of the Transformer, the encoder is responsible for extracting historical traffic state information, and the decoder autoregressively predicts the future traffic state. The multi-head attention mechanism is adopted by both encoder and decoder aims to focus on rich temporal feature patterns. Moreover, key mask technique is used after computing attention matrix to mask the traffic state at missing moments improving the resilience of the model. Extensive experiments on two real-world traffic flow datasets. The results show that RPConvformer achieves the best performance compared to state-of-the-art time series models. Ablation experiments show that considering the local correlation of time series has a higher gain on prediction performance. Random mask experiments show that the model is robust when the historical data is less than 10% missing. In addition, multi-head attention matrix provides further explanation for the dependence between time nodes. RPConvformer as an improved Transformer-based model can provide new ideas for molding temporal dimension in traffic prediction tasks. Our code has been open-sourced at (https://github.com/YanJieWen/RPConvformer).

An unsupervised transfer learning model based on convolutional auto encoder for non-alcoholic steatohepatitis activity scoring and fibrosis staging of liver histopathological images

An unsupervised transfer learning model based on convolutional auto encoder for non-alcoholic steatohepatitis activity scoring and fibrosis staging of liver histopathological images

Unsupervised learning framework for temperature compensated damage identification and localization in ultrasonic guided wave SHM with transfer learning

Damage localization algorithms for ultrasonic guided wave-based structural health monitoring (GW-SHM) typically utilize manually-defined features and supervised machine learning on data collected under various conditions. This scheme has limitations that affect prediction accuracy in practical settings when the model encounters data with a distribution different from that used for training, especially due to variation in environmental factors (e.g., temperature) and types of damages. While deep learning based models that overcome these limitations have been reported in literature, they typically comprise of millions of trainable parameters. As an alternative, we propose an unsupervised approach for temperature-compensated damage identification and localization in GW-SHM systems based on transferring learning from a convolutional auto encoder (TL-CAE). Remarkably, without using signals corresponding to the damages during training (unsupervised), our method demonstrates more accurate damage detection and localization as well as robustness to temperature variations than supervised approaches reported on the publicly available Open Guided Waves (OGW) dataset. Additionally, we have demonstrated reduction in number of trainable parameters using transfer learning (TL) to leverage similarities between time-series among various sensor paths. It is also worth noting that the proposed framework uses raw time-domain signals, without any pre-processing or knowledge of material properties. It should therefore be scalable and adaptable for other materials, structures, damages, and temperature ranges, should more data become available in the future. We present an extensive parametric study to demonstrate feasibility of the proposed method.

Multivariate curve resolution with autoencoders for CARS microspectroscopy

Coherent anti-Stokes Raman scattering (CARS) microspectroscopy is a powerful tool for label-free cell imaging thanks to its ability to acquire a rich amount of information. An important family of operations applied to such data is multivariate curve resolution (MCR). It aims to find main components of a dataset and compute their spectra and concentrations in each pixel. Recently, autoencoders began to be studied to accomplish MCR with dense and convolutional models. However, many questions, like the results variability or the reconstruction metric, remain open and applications are limited to hyperspectral imaging. In this article, we present a nonlinear convolutional encoder combined with a linear decoder to apply MCR to CARS microspectroscopy. We conclude with a study of the result variability induced by the encoder initialization.

Prediction of Flank Wear during Turning of EN8 Steel with Cutting Force Signals Using a Deep Learning Approach

Currently, manufacturing industries focus on intelligent manufacturing. Prediction and monitoring of tool wear are essential in any material removal process, and implementation of a tool condition monitoring system (TCMS) is necessary. This work presents the flank wear prediction during the hard turning of EN8 steel using the deep learning (DL) algorithm. The turning operation is conducted with three levels of selected parameters. CNMG 120408 grade, TiN‐coated cemented carbide tool is used for turning. Cutting force and flank wear are assessed under dry‐cutting conditions. DL algorithms such as adaptive neuro‐fuzzy inference system (ANFIS) and convolutional auto encoder (CAE) are used to predict the flank wear of the single‐point cutting tool. The DL model is developed with turning parameters and cutting force to predict flank wear. The different ANFIS and CAE models are employed to develop the prediction model. Grid‐based ANFIS structure with Gauss membership function performed better than ANFIS models. The ANFIS model’s average testing error of 0.0074011 mm and prediction accuracy of 99.81% are achieved.

Time-Domain Speech Separation Networks With Graph Encoding Auxiliary

End-to-end time-domain speech separation with masking strategy has shown its performance advantage, where a 1-D convolutional layer is used as the speech encoder to encode a sliding window of waveform to a latent feature representation, i.e. an embedding vector. A large window leads to low resolution in the speech processing, on the other hand, a small window offers high resolution but at the expense of high computational cost. In this work, we propose a graph encoding technique to model the fine structural knowledge of speech samples in a window of reasonable size. Specifically, we build a graph representation for each latent representation, and encode the structural details with a graph convolutional network encoder. The encoded graph feature representation complements the original latent feature representation and benefits the separation and reconstruction of speech. Experiments on various models and datasets show that our proposed encoding technique significantly improves the speech quality over other time-domain speech encoders.

SP-DSTS-MIMO Scheme-Aided H.266 for Reliable High Data Rate Mobile Video Communication

With the ever growth of Internet users, video applications, and massive data traffic across the network, there is a higher need for reliable bandwidth-efficient multimedia communication. Versatile Video Coding (VVC/H.266) is finalized in September 2020 providing significantly greater compression efficiency compared to Highest Efficient Video Coding (HEVC) while providing versatile effective use for Ultra-High Definition (HD) videos. This article analyzes the quality performance of convolutional codes, turbo codes and self-concatenated convolutional (SCC) codes based on performance metrics for reliable future video communication. The advent of turbo codes was a significant achievement ever in the era of wireless communication approaching nearly the Shannon limit. Turbo codes are operated by the deployment of an interleaver between two Recursive Systematic Convolutional (RSC) encoders in a parallel fashion. Constituent RSC encoders may be operating on the same or different architectures and code rates. The proposed work utilizes the latest source compression standards H.266 and H.265 encoded standards and Sphere Packing modulation aided differential Space Time Spreading (SP-DSTS) for video transmission in order to provide bandwidth-efficient wireless video communication. Moreover, simulation results show that turbo codes defeat convolutional codes with an averaged Eb/N0 gain of 1.5 dB while convolutional codes outperform compared to SCC codes with an Eb/N0 gain of 3.5 dB at Bit Error Rate (BER) of . The Peak Signal to Noise Ratio (PSNR) results of convolutional codes with the latest source coding standard of H.266 is plotted against convolutional codes with H.265 and it was concluded H.266 outperform with about 6 dB PSNR gain at Eb/N0 value of 4.5 dB.

Toward Semantic Communications: Deep Learning-Based Image Semantic Coding

Semantic communications has received growing interest since it can remarkably reduce the amount of data to be transmitted without missing critical information. Most existing works explore the semantic encoding and transmission for text and apply techniques in Natural Language Processing (NLP) to interpret the meaning of the text. In this paper, we conceive the semantic communications for image data that is much more richer in semantics and bandwidth sensitive. We propose an reinforcement learning based adaptive semantic coding (RL-ASC) approach that encodes images beyond pixel level. Firstly, we define the semantic concept of image data that includes the category, spatial arrangement, and visual feature as the representation unit, and propose a convolutional semantic encoder to extract semantic concepts. Secondly, we propose the image reconstruction criterion that evolves from the traditional pixel similarity to semantic similarity and perceptual performance. Thirdly, we design a novel RL-based semantic bit allocation model, whose reward is the increase in rate-semantic-perceptual performance after encoding a certain semantic concept with adaptive quantization level. Thus, the task-related information is preserved and reconstructed properly while less important data is discarded. Finally, we propose the Generative Adversarial Nets (GANs) based semantic decoder that fuses both locally and globally features via an attention module. Experimental results demonstrate that the proposed RL-ASC is noise robust and could reconstruct visually pleasant and semantic consistent image in low bit rate condition.

Retracted: Rethinking Separable Convolutional Encoders for End‐to‐End Semantic Image Segmentation

Retracted: Rethinking Separable Convolutional Encoders for End‐to‐End Semantic Image Segmentation

Automated hair removal in dermoscopy images using shallow and deep learning neural architectures

Removing hair from digital dermoscopy images is occasionally a necessary step before further analysis is applied to the images. This work considers two machine learning approaches that segment the hair pixels from dermoscopy images. Subsequently, morphological post-processing is applied to refine the segmented hair and an image inpainting algorithm replaces the hair pixels with values based on the surrounding image structures. The first hair segmentation approach combines pixel-wise features extracted using the well-known Gaussian image pyramid with a traditional shallow multilayer perceptron (MLP-ANN), to detect hair pixels in images. The second approach uses a deep neural convolutional Encoder – Decoder (ED) network to segment hair. Both hair segmentation methods (MLP-ANN and ED) are trained with a set of 32 dermoscopy images with manually annotated hair, whereas the MLP-ANN dataset is constructed in a pixel-wise manner.Both proposed methods underwent three different assessments. First a set of 50 images with a-priori known hair is used for hair segmentation evaluation. Secondly, a set of 13 different dermoscopy images with hair added using a suitably trained Generative Adversarial Network -GAN- are used to assess the quality of hair removal that generates the hair-free image, in terms of several error metrics with respect to the original hair-free image. Finally, both proposed hair segmentation methods (MLP-ANN and ED) are applied on a set of 200 hair and hair-free images, which is used for training an image classifier to recognize melanoma against nevi lesions and the improvement in the image classification accuracy is measured. Comparative results against several other state-of-the-art hair removal techniques are also presented.Results show that in terms of hair removal, both the proposed hair removal techniques outperform the best performing of the state-of-the-art methods under comparison, in terms of several error metrics. Considering the effect of hair removal on melanoma image classification, the application of both MLP-ANN and ED increases the accuracy of melanoma classification. In all assessments, the ED was consistently the best performer. The statistical significance of the findings is also established.

Elastic Full-Waveform Inversion Using a Physics-Guided Deep Convolutional Encoder–Decoder

Elastic Full-Waveform Inversion Using a Physics-Guided Deep Convolutional Encoder–Decoder

BCTNet: Bi-Branch Cross-Fusion Transformer for Building Footprint Extraction

Building footprint extraction in remote sensing remains challenging due to the diverse appearances of buildings and confusing scenarios. Recently, researchers have revealed that both the globality and locality are vitally important in building footprint extraction tasks and proposed to incorporate the local context and global long-range dependency in the segmentation models. However, the inadequate integration of the globality and locality still leads to incomplete, fake or missing extraction results. To alleviate these problems, a novel segmentation method named Bi-branch Cross-fusion Transformer Network (BCTNet) is proposed in this study. Two parallel branches of the convolutional encoder branch (CB) and the transformer encoder branch (TB) are designed to extract multi-scale feature maps. A concatenation-then-cross-fusion transformer block (CCTB) is put forward to integrate the locality from the CB and globality from the TB in a cross-fusion way at each stage of the encoding process. Then, an adaptive gating module (AGM) is proposed to gate the feature maps from the CCTB to strengthen the important features while suppressing the irrelevant interference information. After that, the segmentation results can be obtained through a simple decoding process. Comprehensive experiments on two benchmark datasets demonstrate that the proposed BCTNet can achieve superior performance compared to the current state-of-the-art (SOTA) segmentation methods.

Drug-disease association prediction based on end-to-end multi-layer heterogeneous graph convolutional encoders

Traditional drug development in wet labs has long been a costly, cumbersome and error-prone process. Thus, taking advantage of computational power to create algorithmic methods for identifying new treatments for approved drugs is of high importance. Most of the previous works have been strongly relying on the drug's features as well as the similarities between the biological entities to predict potential associations between drugs and diseases. However, the relationships between different biological entities provide crucial information and, unlike previously conducted researches, can be used as the knowledge graph for drug-disease pair interaction prediction tasks. In this study, a novel end-to-end multi-layers heterogeneous graph convolutional encoder (LHGCE) has been proposed to predict candidate associations between drugs and diseases. Firstly, it creates multiple networks between different biological entities, then combines them to build a heterogeneous network as a knowledge graph to capture the topological information of the nodes in the network context. After constructing the network and applying a message-passing operation, the embeddings of the drugs and diseases of each drug-disease pair is extracted and fed into the linear layers to predict the potential interactions. The experimental results based on 5-fold cross-validation showed that LHGCE outperforms state-of-the-art methods in terms of the most evaluation metrics.

Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure

The advent of deep learning in the past decade has significantly helped advance image inpainting. Although achieving promising performance, deep learning-based inpainting algorithms still struggle from the distortion caused by the fusion of structural and contextual features, which are commonly obtained from, respectively, deep and shallow layers of a convolutional encoder. Motivated by this observation, we propose a novel progressive inpainting network that maintains the structural and contextual integrity of a processed image. More specifically, inspired by the Gaussian and Laplacian pyramids, the core of the proposed network is a feature extraction module named GLE. Stacking GLE modules enables the network to extract image features from different image frequency components. This ability is important to maintain structural and contextual integrity, for high frequency components correspond to structural information while low frequency components correspond to contextual information. The proposed network utilizes the GLE features to progressively fill in missing regions in a corrupted image in an iterative manner. Our benchmarking experiments demonstrate that the proposed method achieves clear improvement in performance over many state-of-the-art inpainting algorithms.

ICE: Implicit Coordinate Encoder for Multiple Image Neural Representation.

In recent years, implicit neural representations (INR) have shown their great potential to solve many computer graphics and computer vision problems. With this technique, signals such as 2D images or 3D shapes can be fit by training multi-layer perceptrons (MLP) on continuous functions, providing many advantages over conventional discrete representations. Despite being considered a promising approach to 2D image encoding and compression, the application of INR to image collections remains a challenge, since the number of parameters needed rapidly grow with the number of images. In this paper, we propose a fully implicit approach to INR which drastically reduces the size of MLP models in multiple image representation tasks. We introduce the concept of implicit coordinate encoder (ICE) and show it can be used to scale INR with the image number; specifically, by learning a common feature space between images. Furthermore, we show that our method is valid not only for image collections but also for large (gigapixel) images by applying a "divide-and-conquer" strategy. We propose an auto-encoder deep neural network architecture, with a single ICE (encoder) and multiple MLP (decoders), which are jointly trained following a multi-task learning strategy. We demonstrate the benefits coming from ICE when it is implemented as a one-dimensional convolutional encoder, including a better performance of the downstream MLP models with an order of magnitude fewer parameters. Our method is the first one to make use of convolutional blocks in INR networks, unlike the conventional approach of using MLP architectures only. We show the benefits of ICE in two experimental scenarios: a collection of twenty-four small ( 768×512 ) images (Kodak dataset), and a single large ( 3072×3072 ) image (dwarf planet Pluto), achieving better quality than previous fully-implicit methods, using up to 50% fewer parameters.

Adaptive resource optimization for edge inference with goal-oriented communications

Goal-oriented communications represent an emerging paradigm for efficient and reliable learning at the wireless edge, where only the information relevant for the specific learning task is transmitted to perform inference and/or training. The aim of this paper is to introduce a novel system design and algorithmic framework to enable goal-oriented communications. Specifically, inspired by the information bottleneck principle and targeting an image classification task, we dynamically change the size of the data to be transmitted by exploiting banks of convolutional encoders at the device in order to extract meaningful and parsimonious data features in a totally adaptive and goal-oriented fashion. Exploiting knowledge of the system conditions, such as the channel state and the computation load, such features are dynamically transmitted to an edge server that takes the final decision, based on a proper convolutional classifier. Hinging on Lyapunov stochastic optimization, we devise a novel algorithmic framework that dynamically and jointly optimizes communication, computation, and the convolutional encoder classifier, in order to strike a desired trade-off between energy, latency, and accuracy of the edge learning task. Several simulation results illustrate the effectiveness of the proposed strategy for edge learning with goal-oriented communications.