Robust Representation Learning Research Articles

Optical flow for autonomous driving: Applications, challenges and improvements

Estimating optical flow presents unique challenges in AV applications: large translational motion, wide variations in depth of important objects, strong lens distortion in commonly used fisheye cameras and rolling shutter artefacts in dynamic scenes. Even simple translational motion can produce complicated optical flow fields. Lack of ground truth data also creates a challenge. We evaluate recent optical flow methods on fisheye imagery found in AV applications. We explore various training techniques in challenging scenarios and domain adaptation for transferring models trained on synthetic data where ground truth is available to real-world data. We propose novel strategies that facilitate learning robust representations efficiently to address low-light degeneracies. Finally, we discuss the main challenges and open problems in this problem domain.

Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognition.

In real-world scenarios, collected and annotated data often exhibit the characteristics of multiple classes and long-tailed distribution. Additionally, label noise is inevitable in large-scale annotations and hinders the applications of learning-based models. Although many deep learning based methods have been proposed for handling long-tailed multi-label recognition or label noise respectively, learning with noisy labels in long-tailed multi-label visual data has not been well-studied because of the complexity of long-tailed distribution entangled with multi-label correlation. To tackle such a critical yet thorny problem, this paper focuses on reducing noise based on some inherent properties of multi-label classification and long-tailed learning under noisy cases. In detail, we propose a Stitch-Up augmentation to synthesize a cleaner sample, which directly reduces multi-label noise by stitching up multiple noisy training samples. Equipped with Stitch-Up, a Heterogeneous Co-Learning framework is further designed to leverage the inconsistency between long-tailed and balanced distributions, yielding cleaner labels for more robust representation learning with noisy long-tailed data. To validate our method, we build two challenging benchmarks, named VOC-MLT-Noise and COCO-MLT-Noise, respectively. Extensive experiments are conducted to demonstrate the effectiveness of our proposed method. Compared to a variety of baselines, our method achieves superior results.

Subdomain contraction in deep networks for robust representation learning

Deep neural networks provide end-to-end tools to learn effective representations from data directly. The deep structure makes it possible to model a complicated pattern, even if it has a variety of changes. This leads to a problem that noises and outliers are usually treated as a specific pattern, which is also learned in the network. It is one reason for overfitting incapable of being addressed sufficiently in deep networks. This paper proposes a new method called subdomain contraction (SDC) to tackle the problem. The idea is that our approach inclines to learn more about the shared features between the subsets of the samples but less about the specific features found in only one or two subsets. To this end, the SDC loss penalizes the distribution distance between sub-domains in the feature space to constrain the sharing level of features. By applying the SDC loss term, the data drive the learning process to an optimal tradeoff between modeling noises and the varieties of the pattern. In this manner, the SDC models the pattern as much as possible and ignores most noises, thus improving the generalization ability. The SDC loss can be efficiently computed in minibatches and can also work collaboratively with other regularization methods such as dropout to further improve the performance. Extensive experiments demonstrate that SDC can improve the effectiveness and robustness of representation learning in deep networks against noises, and the superiority is most remarkable with noisy data.

Self-supervised learning methods and applications in medical imaging analysis: a survey.

The scarcity of high-quality annotated medical imaging datasets is a major problem that collides with machine learning applications in the field of medical imaging analysis and impedes its advancement. Self-supervised learning is a recent training paradigm that enables learning robust representations without the need for human annotation which can be considered an effective solution for the scarcity of annotated medical data. This article reviews the state-of-the-art research directions in self-supervised learning approaches for image data with a concentration on their applications in the field of medical imaging analysis. The article covers a set of the most recent self-supervised learning methods from the computer vision field as they are applicable to the medical imaging analysis and categorize them as predictive, generative, and contrastive approaches. Moreover, the article covers 40 of the most recent research papers in the field of self-supervised learning in medical imaging analysis aiming at shedding the light on the recent innovation in the field. Finally, the article concludes with possible future research directions in the field.

Heuristic Attention Representation Learning for Self-Supervised Pretraining.

Recently, self-supervised learning methods have been shown to be very powerful and efficient for yielding robust representation learning by maximizing the similarity across different augmented views in embedding vector space. However, the main challenge is generating different views with random cropping; the semantic feature might exist differently across different views leading to inappropriately maximizing similarity objective. We tackle this problem by introducing Heuristic Attention Representation Learning (HARL). This self-supervised framework relies on the joint embedding architecture in which the two neural networks are trained to produce similar embedding for different augmented views of the same image. HARL framework adopts prior visual object-level attention by generating a heuristic mask proposal for each training image and maximizes the abstract object-level embedding on vector space instead of whole image representation from previous works. As a result, HARL extracts the quality semantic representation from each training sample and outperforms existing self-supervised baselines on several downstream tasks. In addition, we provide efficient techniques based on conventional computer vision and deep learning methods for generating heuristic mask proposals on natural image datasets. Our HARL achieves +1.3% advancement in the ImageNet semi-supervised learning benchmark and +0.9% improvement in AP50 of the COCO object detection task over the previous state-of-the-art method BYOL. Our code implementation is available for both TensorFlow and PyTorch frameworks.

Unsupervised Adversarially Robust Representation Learning on Graphs

Unsupervised/self-supervised pre-training methods for graph representation learning have recently attracted increasing research interests, and they are shown to be able to generalize to various downstream applications. Yet, the adversarial robustness of such pre-trained graph learning models remains largely unexplored. More importantly, most existing defense techniques designed for end-to-end graph representation learning methods require pre-specified label definitions, and thus cannot be directly applied to the pre-training methods. In this paper, we propose an unsupervised defense technique to robustify pre-trained deep graph models, so that the perturbations on the input graph can be successfully identified and blocked before the model is applied to different downstream tasks. Specifically, we introduce a mutual information-based measure, graph representation vulnerability (GRV), to quantify the robustness of graph encoders on the representation space. We then formulate an optimization problem to learn the graph representation by carefully balancing the trade-off between the expressive power and the robustness (i.e., GRV) of the graph encoder. The discrete nature of graph topology and the joint space of graph data make the optimization problem intractable to solve. To handle the above difficulty and to reduce computational expense, we further relax the problem and thus provide an approximate solution. Additionally, we explore a provable connection between the robustness of the unsupervised graph encoder and that of models on downstream tasks. Extensive experiments demonstrate that even without access to labels and tasks, our model is still able to enhance robustness against adversarial attacks on three downstream tasks (node classification, link prediction, and community detection) by an average of +16.5% compared with existing methods.

RRL-GAT: Graph Attention Network-Driven Multilabel Image Robust Representation Learning

Exploring the characterization laws of image data and improving the efficiency of image data characterization knowledge is essential to promote the development of the Internet of Things technology. Considering that images in the real world usually contain multiple objects, and the objects are closely dependent. For these reasons, it brings great challenges to the robust representation learning of multilabel images. In general, researchers model the relationship between objects based on a class activation map and use graph convolution to mine the dependencies between objects. However, graph structure data often contain noise, which means that the edges between nodes are sometimes not so reliable, and the relative importance of neighbors is also different. Based on this, our goal is to reduce noisy connections and false connections between objects, eliminate multilabel image representation bias, and learn robust representations. Therefore, we propose a robust representation learning method for multilabel images driven by graph attention network (RRL-GAT). Specifically, to reduce the accidental false connection of objects in the image, we propose the class attention graph convolution module (C-GAT) to mine the strong association structure between categories. Besides, for the dynamic correlation between objects in the image, we propose an adaptive graph attention convolution module (A-GAT) to capture the subtle dynamic dependencies in the image. The results on two authoritative data sets show that our method is significantly better than all current state-of-the-art methods. Besides, the visualization results show that RRL-GAT can capture the semantic relationship of a specific input image and has sufficient recognizability.

Bidirectional Denoising Autoencoders-Based Robust Representation Learning for Underwater Acoustic Target Signal Denoising

The marine environmental noise formed by wind noise, rain noise, biological noise, sea surface waves, seismic disturbances, and so on is a kind of interference background field in underwater acoustic channels, which brings adverse effects to underwater acoustic target recognition. To improve the recognition accuracy of underwater targets under background noise interference, a bidirectional denoising autoencoder (BDAE) is proposed in this paper for underwater acoustic target signal denoising robust representation learning. The proposed bidirectional denoising autoencoder is an extension of the regular denoising autoencoder, which uses the original underwater acoustic target signals and their corresponding denoised signals to learn robust representations. We then measure the usefulness of the learned representations using a support vector machine classifier. Our proposed approach is verified on the ShipsEar database. Experimental results indicate that the proposed bidirectional denoising autoencoder can effectively learn the robust representations of underwater acoustic target signal denoising and is superior to the traditional methods.

Semi-Supervised Domain Adaptive Structure Learning.

Semi-supervised domain adaptation (SSDA) is quite a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. Unfortunately, a simple combination of domain adaptation (DA) and semi-supervised learning (SSL) methods often fail to address such two objects because of training data bias towards labeled samples. In this paper, we introduce an adaptive structure learning method to regularize the cooperation of SSL and DA. Inspired by the multi-views learning, our proposed framework is composed of a shared feature encoder network and two classifier networks, trained for contradictory purposes. Among them, one of the classifiers is applied to group target features to improve intra-class density, enlarging the gap of categorical clusters for robust representation learning. Meanwhile, the other classifier, serviced as a regularizer, attempts to scatter the source features to enhance the smoothness of the decision boundary. The iterations of target clustering and source expansion make the target features being well-enclosed inside the dilated boundary of the corresponding source points. For the joint address of cross-domain features alignment and partially labeled data learning, we apply the maximum mean discrepancy (MMD) distance minimization and self-training (ST) to project the contradictory structures into a shared view to make the reliable final decision. The experimental results over the standard SSDA benchmarks, including DomainNet and Office-home, demonstrate both the accuracy and robustness of our method over the state-of-the-art approaches.

Towards computationally efficient prediction of molecular signatures from routine histology images

Towards computationally efficient prediction of molecular signatures from routine histology images

Synthetic18K: Learning better representations for person re-ID and attribute recognition from 1.4 million synthetic images

Learning robust representations is critical for the success of person re-identification and attribute recognition systems. However, to achieve this, we must use a large dataset of diverse person images as well as annotations of identity labels and/or a set of different attributes. Apart from the obvious concerns about privacy issues, the manual annotation process is both time consuming and too costly. In this paper, we instead propose to use synthetic person images for addressing these difficulties. Specifically, we first introduce Synthetic18K, a large-scale dataset of over 1 million computer generated person images of 18K unique identities with relevant attributes. Moreover, we demonstrate that pretraining of simple deep architectures on Synthetic18K for person re-identification and attribute recognition and then fine-tuning on real data leads to significant improvements in prediction performances, giving results better than or comparable to state-of-the-art models.

LCC-Net: A Lightweight Cross-Consistency Network for Semisupervised Cardiac MR Image Segmentation.

Semantic segmentation plays a crucial role in cardiac magnetic resonance (MR) image analysis. Although supervised deep learning methods have made significant performance improvements, they highly rely on a large amount of pixel-wise annotated data, which are often unavailable in clinical practices. Besides, top-performing methods usually have a vast number of parameters, which result in high computation complexity for model training and testing. This study addresses cardiac image segmentation in scenarios where few labeled data are available with a lightweight cross-consistency network named LCC-Net. Specifically, to reduce the risk of overfitting on small labeled datasets, we substitute computationally intensive standard convolutions with a lightweight module. To leverage plenty of unlabeled data, we introduce extreme consistency learning, which enforces equivariant constraints on the predictions of different perturbed versions of the input image. Cutting and mixing different training images, as an extreme perturbation on both the labeled and unlabeled data, are utilized to enhance the robust representation learning. Extensive comparisons demonstrate that the proposed model shows promising performance with high annotation- and computation-efficiency. With only two annotated subjects for model training, the LCC-Net obtains a performance gain of 14.4% in the mean Dice over the baseline U-Net trained from scratch.

RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss.

RGBT tracking has attracted increasing attention since RGB and thermal infrared data have strong complementary advantages, which could make trackers all-day and all-weather work. Existing works usually focus on extracting modality-shared or modality-specific information, but the potentials of these two cues are not well explored and exploited in RGBT tracking. In this paper, we propose a novel multi-adapter network to jointly perform modality-shared, modality-specific and instance-aware target representation learning for RGBT tracking. To this end, we design three kinds of adapters within an end-to-end deep learning framework. In specific, we use the modified VGG-M as the generality adapter to extract the modality-shared target representations. To extract the modality-specific features while reducing the computational complexity, we design a modality adapter, which adds a small block to the generality adapter in each layer and each modality in a parallel manner. Such a design could learn multilevel modality-specific representations with a modest number of parameters as the vast majority of parameters are shared with the generality adapter. We also design instance adapter to capture the appearance properties and temporal variations of a certain target. Moreover, to enhance the shared and specific features, we employ the loss of multiple kernel maximum mean discrepancy to measure the distribution divergence of different modal features and integrate it into each layer for more robust representation learning. Extensive experiments on two RGBT tracking benchmark datasets demonstrate the outstanding performance of the proposed tracker against the state-of-the-art methods.

TEST: Triplet Ensemble Student-Teacher Model for Unsupervised Person Re-Identification.

The self-ensembling methods have achieved amazing performance for semi-supervised representation learning and domain adaptation. However, the disadvantage of these methods is that the teacher network is tightly coupled with the student network, which limits the descriptive ability of the self-ensembling model. To overcome the coupling effect between the teacher network and the student network, we propose a novel Triplet Ensemble Student-Teacher (TEST) model for unsupervised person re-identification, which consists of one teacher network T and two student networks S1 and S2 . Similar to the traditional self-ensembling model, the student network S1 is applied to update the teacher network T . Furthermore, a closed-loop learning mechanism is built in the TEST model by imposing an ensemble consistent constraint between T and S2 , and performing a heterogeneous co-teaching procedure between S1 and S2 . With the closed-loop learning mechanism, the TEST model can loosen the constraint between the teacher T and the student S1 , and enhance the descriptive ability of S1 . Besides, the knowledge exchange between S1 and S2 can ensure that the two student networks can elegantly deal with the noisy labels and avoid coupling. By training the TEST model with the clustering-generated pseudo labels, we can achieve effective and robust representation learning for unsupervised person re-identification. The evaluations on three widely-used benchmarks show that our approach can achieve significant performance compared with state-of-the-art methods.

Unsupervised representation learning by discovering reliable image relations

Learning robust representations that allow to reliably establish relations between images is of paramount importance for virtually all of computer vision. Annotating the quadratic number of pairwise relations between training images is simply not feasible, while unsupervised inference is prone to noise, thus leaving the vast majority of these relations to be unreliable. To nevertheless find those relations which can be reliably utilized for learning, we follow a divide-and-conquer strategy: We find reliable similarities by extracting compact groups of images and reliable dissimilarities by partitioning these groups into subsets, converting the complicated overall problem into few reliable local subproblems. For each of the subsets we obtain a representation by learning a mapping to a target feature space so that their reliable relations are kept. Transitivity relations between the subsets are then exploited to consolidate the local solutions into a concerted global representation. While iterating between grouping, partitioning, and learning, we can successively use more and more reliable relations which, in turn, improves our image representation. In experiments, our approach shows state-of-the-art performance on unsupervised classification on ImageNet with 46.0% and competes favorably on different transfer learning tasks on PASCAL VOC.

Latent correlation embedded discriminative multi-modal data fusion

Ever-growing volume of multi-modal data has garnered considerable research enthusiasm in many fields, such as biomedical image analysis and biometrics. It remains a challenge how to effectively fuse multi-modal data for improving the learning performance. Many data fusion methods have been proposed, such as multi-kernel learning (MKL) and canonical component analysis (CCA). Most of the previous fusion methods focus on concatenating the multi-modal data, which fail to preserve the structure information across different modalities. In addition, they treat the samples equally in fusion and ignore the difference of the contributions of samples to the fusion model. In this paper, we proposed a discriminative multi-modal dimensionality reduction method, which can seamlessly fuse the multi-modal data and explore the latent correlation among different modalities for robust representation learning. In optimization, self-paced learning is adopted to dynamically estimate the contribution of each sample to the overall fusion model. Following the easy-to-hard learning sequence, our model can adaptively and sequentially enhance the robustness of the learning system in a knowledge discovery manner. Extensive experimental results on multi-modal brain disease diagnosis and multi-spectral palmprint classification tasks demonstrate the proposed method outperforms the previous multi-modal classification methods.

Exemplar-Supported Representation for Effective Class-Incremental Learning

Catastrophic forgetting is a key challenge for class-incremental learning with deep neural networks, where the performance decreases considerably while dealing with long sequences of new classes. To tackle this issue, in this paper, we propose a new exemplar-supported representation for incremental learning (ESRIL) approach that consists of three components. First, we use memory aware synapses (MAS) pre-trained on the ImageNet to retain the ability of robust representation learning and classification for old classes from the perspective of the model. Second, exemplar-based subspace clustering (ESC) is utilized to construct the exemplar set, which can keep the performance from various views of the data. Third, the nearest class multiple centroids (NCMC) is used as the classifier to save the training cost of the fully connected layer of MAS when the criterion is met. Intensive experiments and analyses are presented to show the influence of various backbone structures and the effectiveness of different components in our model. Experiments on several general-purpose and fine-grained image recognition datasets have fully demonstrated the efficacy of the proposed methodology.

Domain adaptive representation learning for facial action unit recognition

Learning robust representations for applications with multiple modalities of input can have a significant impact on improving performance. Traditional representation learning methods rely on projecting the input modalities to a common subspace to maximize agreement amongst the modalities for a particular task. We propose a novel approach to representation learning that uses a latent representation decoder to reconstruct the target modality and thereby employ the target modality purely as a supervision signal for discovering correlations between the modalities. Through cross modality supervision, we demonstrate that the learnt representation is able to improve upon the performance of the task of facial action unit (AU) recognition over modality specific representations and even their fused counterparts. As an extension, we explore a new transfer learning technique to adapt the learnt representation to the target domain. We also present a shared representation based feature fusion methodology to improve the performance of any multi-modal system. Our experiments on three AU recognition datasets - MMSE, BP4D and DISFA, show strong performance gains producing state-of-the-art results in spite of the absence of data from a modality.

Simultaneous multiple features tracking of beats: A representation learning approach to reduce false alarm rates in ICUs.

The high rate of false alarms is a key challenge related to patient care in intensive care units (ICUs) that can result in delayed responses of the medical staff. Several rule-based and machine learning-based techniques have been developed to address this problem. However, the majority of these methods rely on the availability of different physiological signals such as different electrocardiogram (ECG) leads, arterial blood pressure (ABP), and photoplethysmogram (PPG), where each signal is analyzed by an independent processing unit and the results are fed to an algorithm to determine an alarm. That calls for novel methods that can accurately detect the cardiac events by only accessing one signal (e.g., ECG) with a low level of computation and sensors requirement. We propose a novel and robust representation learning framework for ECG analysis that only rely on a single lead ECG signal and yet achieves considerably better performance compared to the state-of-the-art works in this domain, without relying on an expert knowledge. We evaluate the performance of this method using the "2015 Physionet computing in cardiology challenge" dataset. To the best of our knowledge, the best previously reported performance is based on both expert knowledge and machine learning where all available signals of ECG, ABP and PPG are utilized. Our proposed method reaches the performance of 97.3%, 95.5 %, and 90.8 % in terms of sensitivity, specificity, and the challenge's score, respectively for the detection of five arrhythmias when only one single ECG lead signals is used without any expert knowledge.

Adversarial Network Embedding

Learning low-dimensional representations of networks has proved effective in a variety of tasks such as node classification, link prediction and network visualization. Existing methods can effectively encode different structural properties into the representations, such as neighborhood connectivity patterns, global structural role similarities and other high-order proximities. However, except for objectives to capture network structural properties, most of them suffer from lack of additional constraints for enhancing the robustness of representations. In this paper, we aim to exploit the strengths of generative adversarial networks in capturing latent features, and investigate its contribution in learning stable and robust graph representations. Specifically, we propose an Adversarial Network Embedding (ANE) framework, which leverages the adversarial learning principle to regularize the representation learning. It consists of two components, i.e., a structure preserving component and an adversarial learning component. The former component aims to capture network structural properties, while the latter contributes to learning robust representations by matching the posterior distribution of the latent representations to given priors. As shown by the empirical results, our method is competitive with or superior to state-of-the-art approaches on benchmark network embedding tasks.