Deep Sparse Representation Research Articles

Knowledge transduction for cross-domain few-shot learning

Cross-Domain Few-Shot Learning (CDFSL) aims to classify new categories from new domains with few samples. It confronts a greater domain shift than Few-Shot Learning (FSL). Based on the transfer learning framework, we propose a Knowledge Transduction method (KT) to alleviate domain shift and achieve few-shot recognition. First, a feature adaptation module based on feed-forward attention is constructed to learn domain-adapted features. The feature adaptation module weakens domain shift by transducing knowledge from an auxiliary dataset to the new dataset. Second, a feature transduction module based on deep sparse representation is developed to gather class semantics from limited support images. The feature transduction module transduces knowledge from support images to query images for few-shot recognition. In addition, a stochastic image augmentation method is proposed for FSL to train a more generalized model through consistency representation learning. Our method achieves competitive accuracy on four CDFSL datasets and four FSL datasets compared to state-of-the-art methods. The source code is available at https://github.com/XDUpfLi/KT.

A Deep Sparse Representation with Random Dictionary for Hyperspectral Image Classification

Hyperspectral image (HSI) classification methods based on deep learning have demonstrated excellent performance, while these deep learning methods take a lot of time to train the parameters. In this paper, we propose a deep sparse representation (SR) network (DSRNet) without spending a lot of time training network parameters in the feature extraction stage. The contributions of this paper are three-fold. First, we introduce random dictionary into HSI classification, and solve sparse representation model under this dictionary. Second, we extend the shallow sparse representation model to the deep sparse representation model, where the SR model needs to be solved for each layer and used to extract the deep features of HSI. Finally, we investigate the classification performance of different classifiers on the deep features extracted by using DSRNet. Experimental results show that the proposed method can achieve better classification results compared with some closely related HSI classification methods and the other state-of-the-art deep learning methods.

Double Sparse Deep Reinforcement Learning via Multilayer Sparse Coding and Nonconvex Regularized Pruning.

Deep reinforcement learning (DRL), which highly depends on the data representation, has shown its potential in many practical decision-making problems. However, the process of acquiring representations in DRL is easily affected by interference from models, and moreover leaves unnecessary parameters, leading to control performance reduction. In this article, we propose a double sparse DRL via multilayer sparse coding and nonconvex regularized pruning. To alleviate interference in DRL, we propose a multilayer sparse-coding-structural network to obtain deep sparse representation for control in reinforcement learning. Furthermore, we employ a nonconvex log regularizer to promote strong sparsity, efficiently removing the unnecessary weights with a regularizer-based pruning scheme. Hence, a double sparse DRL algorithm is developed, which can not only learn deep sparse representation to reduce the interference but also remove redundant weights while keeping the robust performance. The experimental results in five benchmark environments of the deep q network (DQN) architecture demonstrate that the proposed method with deep sparse representations from the multilayer sparse-coding structure can outperform existing sparse-coding-based DRL in control, for example, completing Mountain Car with 140.81 steps, achieving near 10% reward increase from the single-layer sparse-coding DRL algorithm, and obtaining 286.08 scores in Catcher, which are over two times the rewards of the other algorithms. Moreover, the proposed algorithm can reduce over 80% parameters while keeping performance improvements from deep sparse representations.

Deep Sparse Representation Based Image Restoration With Denoising Prior

As a powerful statistical signal modeling technique, sparse representation has been widely used in various image restoration (IR) applications. The sparsity-based methods have achieved leading performance in the past few decades. However, in recent years it has been surpassed by other methods, especially the recent deep learning based methods. In this paper, we address the question that whether sparse representation can be competitive again. The way we answer this question is to redesign it with a deep architecture. To be specific, we propose an end-to-end deep architecture that follows the process of the sparse representation based IR. In particular, we learn a sparse convolutional dictionary to replace the traditional dictionary, and a convolutional neural network (CNN) denoising prior to replace the image prior. Through end-to-end training, the parameters in convolutional dictionary and CNN denoiser can be jointly optimized. Experimental results on several representative IR tasks, including image denoising, deblurring and super-resolution, demonstrate that the proposed deep network can achieve superior performance against state-of-the-art model-based and learning-based methods.

DSRD: deep sparse representation with learnable dictionary for remotely sensed image denoising

ABSTRACT Patch-based modelling methods (i.e. sparse coding) had proven their great ability in solving the degradation problem caused by additive white Gaussian noise (AWGN) in remotely sensing images. However, these methods usually pursue surprising performance with sacrificing computational efficiency, and the learned dictionary is universal, resulting in a weak model representation flexibility. To address these issues, this paper proposes a deep sparse representation with learnable dictionary (DSRD) scheme, where the major difference from the previous sparse coding methods is that sparse representation coefficients and dictionaries are both deeply learned, acted as two modular parts to be plugged into the unfolded sparse coding model, to speed up for a stable solution as well as yielding competitive performance. The ablation studies illustrate that our DSRD strategy is efficient for a fixed solution while a flexible and powerful capacity in producing pleasing denoising performance. Meanwhile, the comparisons of quantitatively and qualitatively experimental results further demonstrate that it is effective and can generate enjoyable denoising performance that even outperforms that produced by other state-of-the-arts.

Deep sparse representation network for feature learning of vibration signals and its application in gearbox fault diagnosis

Vibration signals play a key role in machinery fault diagnosis, which are often buried by strong noises due to complex working conditions. Typical deep neural networks (e.g., convolutional neural networks (CNNs)) can effectively learn features from vibration data and perform well in machinery fault diagnosis. Nonetheless, the vibration signals collected from real industry are often noised and nonstationary. It is still difficult for CNNs to implement feature extraction and noise reduction from large number of vibration signals. In this paper, a novel deep learning model, deep sparse representation network (DSRNet) is proposed to suppress noise and learn effective features from noised signals directly. Firstly, a sparse representation layer is developed to extract impulsive components and suppress noise hidden in vibration signals in an end-to-end manner. Secondly, an adaptive densely stacked convolutional structure is proposed to extract effective features from the filtered signals by the sparse representation layer. Finally, the effectiveness of DSRNet for feature learning on vibration signals is verified on two gearbox cases. The experimental results show that DSRNet has good feature learning and signal denoising performance, which outperforms those famous CNNs (e.g., ResNet, DenseNet).

Graph Regularized Deep Sparse Representation for Unsupervised Anomaly Detection.

Anomaly detection (AD) aims to distinguish the data points that are inconsistent with the overall pattern of the data. Recently, unsupervised anomaly detection methods have aroused huge attention. Among these methods, feature representation (FR) plays an important role, which can directly affect the performance of anomaly detection. Sparse representation (SR) can be regarded as one of matrix factorization (MF) methods, which is a powerful tool for FR. However, there are some limitations in the original SR. On the one hand, it just learns the shallow feature representations, which leads to the poor performance for anomaly detection. On the other hand, the local geometry structure information of data is ignored. To address these shortcomings, a graph regularized deep sparse representation (GRDSR) approach is proposed for unsupervised anomaly detection in this work. In GRDSR, a deep representation framework is first designed by extending the single layer MF to a multilayer MF for extracting hierarchical structure from the original data. Next, a graph regularization term is introduced to capture the intrinsic local geometric structure information of the original data during the process of FR, making the deep features preserve the neighborhood relationship well. Then, a L1-norm-based sparsity constraint is added to enhance the discriminant ability of the deep features. Finally, a reconstruction error is applied to distinguish anomalies. In order to demonstrate the effectiveness of the proposed approach, we conduct extensive experiments on ten datasets. Compared with the state-of-the-art methods, the proposed approach can achieve the best performance.

Deep Sparse Representation Classifier for facial recognition and detection system

This paper proposes a two-layer Convolutional Neural Network (CNN) to learn the high-level features which utilizes to the face identification via sparse representation. Feature extraction plays a vital role in real-world pattern recognition and classification tasks. The details description of the given input face image, significantly improve the performance of the facial recognition system. Sparse Representation Classifier (SRC) is a popular face classifier that sparsely represents the face image by a subset of training data, which is known as insensitive to the choice of feature space. The proposed method shows the performance improvement of SRC via a precisely selected feature exactor. The experimental results show that the proposed method outperforms other methods on given datasets.

Facial age feature extraction based on deep sparse representation

In recent years, the research on facial age estimation has attracted more and more interests from the scholars and researchers in the field of psychological, aesthetic, forensic science and computer vision. However, facial age estimation is a challenging work: face age does not suffer from only the influence of intrinsic factors (e.g., genes) but also external factors (e.g., living conditions), so that it is difficult to find accurate features which can describe the change of age. Therefore, this paper proposes a robust face feature extraction method based on dynamic deep sparse representation. Combined with the respective characteristics of Active Appearance Model (AAM), Local Binary Patterns (LBP), Gabor and Bio-Inspired Features (BIF), this method will give sufficient consideration to the thinking way of object recognition of the human, similarity of adjacent ages and the classification principle of signal sparse representation. In addition, in order to reduce the interference of the face identity factor, two factors analysis method is proposed to separate the face identity factor. The experimental results show that the feature extraction method proposed in this paper has strong discrimination and robustness, which outperforms the state-of-the-art age estimation approaches.

Deep-Sparse-Representation-Based Features for Speech Recognition

Features derived using sparse representation (SR)-based approaches have been shown to yield promising results for speech recognition tasks. In most of the approaches, the SR corresponding to speech signal is estimated using a dictionary, which could be either exemplar based or learned. However, a single-level decomposition may not be suitable for the speech signal, as it contains complex hierarchical information about various hidden attributes. In this paper, we propose to use a multilevel decomposition (having multiple layers), also known as the deep sparse representation (DSR), to derive a feature representation for speech recognition. Instead of having a series of sparse layers, the proposed framework employs a dense layer between two sparse layers, which helps in efficient implementation. Our studies reveal that the representations obtained at different sparse layers of the proposed DSR model have complimentary information. Thus, the final feature representation is derived after concatenating the representations obtained at the sparse layers. This results in a more discriminative representation, and improves the speech recognition performance. Since the concatenation results in a high-dimensional feature, principal component analysis is used to reduce the dimension of the obtained feature. Experimental studies demonstrate that the proposed feature outperforms existing features for various speech recognition tasks.

Robust multi-atlas label propagation by deep sparse representation

Robust multi-atlas label propagation by deep sparse representation

Robust multi-atlas label propagation by deep sparse representation

Recently, multi-atlas patch-based label fusion has achieved many successes in medical imaging area. The basic assumption in the current state-of-the-art approaches is that the image patch at the target image point can be represented by a patch dictionary consisting of atlas patches from registered atlas images. Therefore, the label at the target image point can be determined by fusing labels of atlas image patches with similar anatomical structures. However, such assumption on image patch representation does not always hold in label fusion since (1) the image content within the patch may be corrupted due to noise and artifact; and (2) the distribution of morphometric patterns among atlas patches might be unbalanced such that the majority patterns can dominate label fusion result over other minority patterns. The violation of the above basic assumptions could significantly undermine the label fusion accuracy. To overcome these issues, we first consider forming label-specific group for the atlas patches with the same label. Then, we alter the conventional flat and shallow dictionary to a deep multi-layer structure, where the top layer (label-specific dictionaries) consists of groups of representative atlas patches and the subsequent layers (residual dictionaries) hierarchically encode the patchwise residual information in different scales. Thus, the label fusion follows the representation consensus across representative dictionaries. However, the representation of target patch in each group is iteratively optimized by using the representative atlas patches in each label-specific dictionary exclusively to match the principal patterns and also using all residual patterns across groups collaboratively to overcome the issue that some groups might be absent of certain variation patterns presented in the target image patch. Promising segmentation results have been achieved in labeling hippocampus on ADNI dataset, as well as basal ganglia and brainstem structures, compared to other counterpart label fusion methods.