Deep Encoder-decoder Architecture Research Articles

Enhanced deep learning network for accurate digital elevation model generation from LiDAR data

This paper presents a unique hierarchical deep network to tackle the task of identifying and filtering non-ground objects from point cloud data. This task is essential in the building of digital terrain models (DTMs). The proposed network is based on a deep encoder-decoder architecture and includes efficient convolutional connections to improve the identification of items that are not on the ground. In this architectural framework, a block for extracting features is intentionally created to capture a wide range of characteristics at many levels. Additionally, a technique for fusing global and local data is included to further enhance the accuracy of detection. The effectiveness of the proposed deep network in accurately detecting objects is validated by a comparative study with current approaches, utilizing ISPRS data. This analysis demonstrates the superiority of the proposed network in terms of object detection accuracy.

A deep contrastive framework for unsupervised temporal link prediction in dynamic networks

In dynamic networks, temporal link prediction aims to predict the appearance and disappearance of links in future snapshots based on the network structure we have observed. It also plays a crucial role in network analysis and predicting the behavior of the dynamic system. However, most existing studies only focus on supervised temporal link prediction problems, i.e., taking part of the links in future snapshots as supervised information. The ones that can solve the unsupervised temporal link prediction problem are mainly based on matrix decomposition, which lack the capability to automatically extract nonlinear spatial and temporal features from dynamic networks. The most challenging part of this problem is to extract the inherent evolution of the patterns hidden in dynamic networks in unsupervised ways. Inspired by the application and achievement of contrastive learning in network representation learning, we propose a novel deep Contrastive framework for unsupervised Temporal Link Prediction (CTLP). Our framework is based on a deep encoder-decoder architecture, which can capture the nonlinear structure and temporal features automatically and can predict future links of subsequent snapshots of dynamic networks in an unsupervised manner. Besides, CTLP could handle the multi-step temporal link prediction problem of dynamic networks through attenuation modeling across the snapshots. Extensive experiments on temporal link prediction show that our CTLP framework significantly outperforms state-of-the-art unsupervised methods, and even outperforms the supervised methods in some cases.

Hourglass 3D CNN for Stereo Disparity Estimation for Mobile Robots

Stereo cameras allow mobile robots to perceive depth in their surroundings by capturing two separate images from slightly different perspectives. This is necessary for tasks such as obstacle avoidance, navigation, and spatial mapping. By utilizing a convolutional neural network (CNN), existing works in stereo cameras based on depth estimation have achieved superior results. However, the critical requirement for depth estimation for mobile robots is to have an optimal tradeoff between computational cost and accuracy. To achieve such a tradeoff, attention-aware feature aggregation (AAFS) has been proposed for real-time stereo matching on edge devices. AAFS includes multistage feature extraction, an attention module, and a 3D CNN architecture. However, its 3D CNN architecture learns contextual information ineffectively. In this paper, a deep encoder–decoder architecture is applied to an AAFS 3D CNN to improve depth estimation accuracy. Through evaluation, it is proven that the proposed 3D CNN architecture provides significantly better accuracy while keeping the inference time comparable to that of AAFS.

WaSR-A Water Segmentation and Refinement Maritime Obstacle Detection Network.

Obstacle detection using semantic segmentation has become an established approach in autonomous vehicles. However, existing segmentation methods, primarily developed for ground vehicles, are inadequate in an aquatic environment as they produce many false positive (FP) detections in the presence of water reflections and wakes. We propose a novel deep encoder-decoder architecture, a water segmentation and refinement (WaSR) network, specifically designed for the marine environment to address these issues. A deep encoder based on ResNet101 with atrous convolutions enables the extraction of rich visual features, while a novel decoder gradually fuses them with inertial information from the inertial measurement unit (IMU). The inertial information greatly improves the segmentation accuracy of the water component in the presence of visual ambiguities, such as fog on the horizon. Furthermore, a novel loss function for semantic separation is proposed to enforce the separation of different semantic components to increase the robustness of the segmentation. We investigate different loss variants and observe a significant reduction in FPs and an increase in true positives (TPs). Experimental results show that WaSR outperforms the current state of the art by approximately 4% in F1 score on a challenging unmanned surface vehicle dataset. WaSR shows remarkable generalization capabilities and outperforms the state of the art by over 24% in F1 score on a strict domain generalization experiment.

A Comparative Study of Various Deep Learning Approaches to Shape Encoding of Planar Geospatial Objects

The shape encoding of geospatial objects is a key problem in the fields of cartography and geoscience. Although traditional geometric-based methods have made great progress, deep learning techniques offer a development opportunity for this classical problem. In this study, a shape encoding framework based on a deep encoder–decoder architecture was proposed, and three different methods for encoding planar geospatial shapes, namely GraphNet, SeqNet, and PixelNet methods, were constructed based on raster-based, graph-based, and sequence-based modeling for shape. The three methods were compared with the existing deep learning-based shape encoding method and two traditional geometric methods. Quantitative evaluation and visual inspection led to the following conclusions: (1) The deep encoder–decoder methods can effectively compute shape features and obtain meaningful shape coding to support the shape measure and retrieval task. (2) Compared with the traditional Fourier transform and turning function methods, the deep encoder–decoder methods showed certain advantages. (3) Compared with the SeqNet and PixelNet methods, GraphNet performed better due to the use of a graph to model the topological relations between nodes and efficient graph convolution and pooling operations to process the node features.

SAEP: A Surrounding-Aware Individual Emotion Prediction Model Combined with T-LSTM and Memory Attention Mechanism

The future emotion prediction of users on social media has been attracting increasing attention from academics. Previous studies on predicting future emotion have focused on the characteristics of individuals’ emotion changes; however, the role of the individual’s neighbors has not yet been thoroughly researched. To fill this gap, a surrounding-aware individual emotion prediction model (SAEP) based on a deep encoder–decoder architecture is proposed to predict individuals’ future emotions. In particular, two memory-based attention networks are constructed: The time-evolving attention network and the surrounding attention network to extract the features of the emotional changes of users and neighbors, respectively. Then, these features are incorporated into the emotion prediction task. In addition, a novel variant LSTM is introduced as the encoder of the proposed model, which can effectively extract complex patterns of users’ emotional changes from irregular time series. Extensive experimental results show that the proposed approach outperforms five alternative methods. The SAEP approach has improved by approximately 4.21–14.84% micro F1 on a dataset built from Twitter and 7.30–13.41% on a dataset built from Microblog. Further analyses validate the effectiveness of the proposed time-evolving context and surrounding context, as well as the factors that may affect the prediction results.

Opinion-unaware blind picture quality measurement using deep encoder–decoder architecture

Recently, deep-learning-based blind picture quality measurement (BPQM) metrics have gained significant attention. However, training a robust deep BPQM metric remains a difficult and challenging task because of the limited number of subject-rated training samples. State-of-the-art full-reference (FR) picture quality measurement (PQM) metrics are in good agreement with human subjective quality scores. Therefore, they can be employed to approximate human subjective quality scores to train BPQM metrics. Inspired by this, we propose a deep encoder–decoder architecture (DEDA) for opinion-unaware (OU) BPQM that does not require human-labeled distorted samples for training. In the training procedure, to avoid overfitting and to ensure the independency of the training and testing samples, we first construct 6,000 distorted pictures and use their objective quality/similarity maps obtained using a high-performance FR-PQM metric for distorted pictures as training labels. Subsequently, an end-to-end map between the distorted pictures and their objective quality/similarity maps (labels) is learned, represented as the DEDA that takes the distorted picture as the input and outputs its predicted quality/similarity map. In the DEDA, the pyramid supervision training strategy is used, which applies supervised learning over three scale layers to efficiently optimize the parameters. In the testing procedure, the quality/similarity maps of the testing samples that can help localize distortions can be predicted with the trained DEDA architecture. The predicted quality/similarity maps are then gradually pooled together to obtain the overall objective quality scores. Comparative experiments on three publicly available standard PQM datasets demonstrate that our proposed DEDA metric is in good agreement with subjective assessment compared to previous state-of-the-art OU-BPQM metrics.

Machine Learning to Improve Multi-Hop Searching and Extended Wireless Reachability in V2X

Multi-hop relay selection is a critical issue in vehicle-to-everything networks. In previous works, the optimal hopping strategy is assumed to be based on the shortest distance. This study proposes a hopping strategy based on the lowest propagation loss, considering the effect of the environment. We use a two-step machine learning routine: improved deep encoder-decoder architecture to generate environmental maps and ${Q}$ -learning to search for the multi-hopping path with the lowest propagation loss. Simulation results show that our proposed method can improve environmental recognition and extend the reachability of multi-hop communications by up to 66.7%, compared with a shortest-distance selection.

Dual Encoder-Decoder Based Generative Adversarial Networks for Disentangled Facial Representation Learning

To learn disentangled representations of facial images, we present a Dual Encoder-Decoder based Generative Adversarial Network (DED-GAN). In the proposed method, both the generator and discriminator are designed with deep encoder-decoder architectures as their backbones. To be more specific, the encoder-decoder structured generator is used to learn a pose disentangled face representation, and the encoder-decoder structured discriminator is tasked to perform real/fake classification, face reconstruction, determining identity and estimating face pose. We further improve the proposed network architecture by minimising the additional pixel-wise loss defined by the Wasserstein distance at the output of the discriminator so that the adversarial framework can be better trained. Additionally, we consider face pose variation to be continuous, rather than discrete in existing literature, to inject richer pose information into our model. The pose estimation task is formulated as a regression problem, which helps to disentangle identity information from pose variations. The proposed network is evaluated on the tasks of pose-invariant face recognition (PIFR) and face synthesis across poses. An extensive quantitative and qualitative evaluation carried out on several controlled and in-the-wild benchmarking datasets demonstrates the superiority of the proposed DED-GAN method over the state-of-the-art approaches.

A Sparse Transformer-Based Approach for Image Captioning

Image Captioning is the task of providing a natural language description for an image. It has caught significant amounts of attention from both computer vision and natural language processing communities. Most image captioning models adopt deep encoder-decoder architectures to achieve state-of-the-art performances. However, it is difficult to model knowledge on relationships between input image region pairs in the encoder. Furthermore, the word in the decoder hardly knows the correlation to specific image regions. In this article, a novel deep encoder-decoder model is proposed for image captioning which is developed on sparse Transformer framework. The encoder adopts a multi-level representation of image features based on self-attention to exploit low-level and high-level features, naturally the correlations between image region pairs are adequately modeled as self-attention operation can be seen as a way of encoding pairwise relationships. The decoder improves the concentration of multi-head self-attention on the global context by explicitly selecting the most relevant segments at each row of the attention matrix. It can help the model focus on the more contributing image regions and generate more accurate words in the context. Experiments demonstrate that our model outperforms previous methods and achieves higher performance on MSCOCO and Flickr30k datasets. Our code is available at https://github.com/2014gaokao/ImageCaptioning .

Low-Complexity Vector Quantized Compressed Sensing via Deep Neural Networks

Sparse signals, encountered in many wireless and signal acquisition applications, can be acquired via compressed sensing (CS) to reduce computations and transmissions, crucial for resource-limited devices, e.g., wireless sensors. Since the information signals are often continuous-valued, digital communication of compressive measurements requires quantization. In such a quantized compressed sensing (QCS) context, we address remote acquisition of a sparse source through vector quantized noisy compressive measurements. We propose a deep encoder-decoder architecture, consisting of an encoder deep neural network (DNN), a quantizer, and a decoder DNN, that realizes low-complexity vector quantization aiming at minimizing the mean-square error of the signal reconstruction for a given quantization rate. We devise a supervised learning method using stochastic gradient descent and backpropagation to train the system blocks. Strategies to overcome the vanishing gradient problem are proposed. Simulation results show that the proposed non-iterative DNN-based QCS method achieves higher rate-distortion performance with lower algorithm complexity as compared to standard QCS methods, conducive to delay-sensitive applications with large-scale signals.

Deep-learning-based inverse design model for intelligent discovery of organic molecules

The discovery of high-performance functional materials is crucial for overcoming technical issues in modern industries. Extensive efforts have been devoted toward accelerating and facilitating this process, not only experimentally but also from the viewpoint of materials design. Recently, machine learning has attracted considerable attention, as it can provide rational guidelines for efficient material exploration without time-consuming iterations or prior human knowledge. In this regard, here we develop an inverse design model based on a deep encoder-decoder architecture for targeted molecular design. Inspired by neural machine language translation, the deep neural network encoder extracts hidden features between molecular structures and their material properties, while the recurrent neural network decoder reconstructs the extracted features into new molecular structures having the target properties. In material design tasks, the proposed fully data-driven methodology successfully learned design rules from the given databases and generated promising light-absorbing molecules and host materials for a phosphorescent organic light-emitting diode by creating new ligands and combinatorial rules.