Linear Combination Of Training Samples Research Articles

Consensus Sparsity: Multi-context Sparse Image Representation via L∞-induced Matrix Variate.

The sparsity is an attractive property that has been widely and intensively utilized in various image processing fields (e.g., robust image representation, image compression, image analysis, etc.). Its actual success owes to the exhaustive mining of the intrinsic (or homogenous) information from the whole data carrying redundant information. From the perspective of image representation, the sparsity can successfully find an underlying homogenous subspace from a collection of training data to represent a given test sample. The famous sparse representation (SR) and its variants embed the sparsity by representing the test sample using a linear combination of training samples with L0-norm regularization and L1-norm regularization. However, although these state-of-the-art methods achieve powerful and robust performances, the sparsity is not fully exploited on the image representation in the following three aspects: 1) the within-sample sparsity, 2) the between-sample sparsity, and 3) the image structural sparsity. In this paper, to make the above-mentioned multi-context sparsity properties agree and simultaneously learned in one model, we propose the concept of consensus sparsity (Con-sparsity) and correspondingly build a multi-context sparse image representation (MCSIR) framework to realize this. We theoretically prove that the consensus sparsity can be achieved by the L∞-induced matrix variate based on the Bayesian inference. Extensive experiments and comparisons with the state-of-the-art methods (including deep learning) are performed to demonstrate the promising performance and property of the proposed consensus sparsity.

A sparse representation denoising algorithm for finger-vein image based on dictionary learning

As an important method of biometric authentication, finger-vein recognition utilizes the unique finger-vein patterns to identify individuals at a high level of accuracy and safety. However, noise components, inherent in finger-vein images, pose a formidable challenge for extracting reliable finger-vein features for recognition. To tackle this challenge, intensive efforts have been directed at sparse representation (SR) methods, which can find the best representative of a test sample by a sparse linear combination of training samples (atoms) from a dictionary. Previous SR approaches treat training atoms equally for image representations, even if these atoms may vary in their effectiveness as feature descriptors, thus jeopardizing the denoising and recognition performances. To overcome this limitation, the present study proposed an adaptive SR with distance-based dictionary learning (DDL), enabling the ability to target more informative training samples. Specifically, based on the Euclidean distance, atoms in the dictionary are classified into two groups: the high-information and the low-information. Their weights for feature representations are assigned based on the distance entropy. Experimental results indicate that the developed SR-DDL denoising method, can suppress image noises and subsequently enhance the image recognition performance.

Ship Detection in SAR Images via Enhanced Nonnegative Sparse Locality-Representation of Fisher Vectors

As a powerful coding strategy for superpixels in synthetic aperture radar (SAR) images, Fisher vector (FV) lies in a low-dimensional subspace and can be sparsely represented as a linear combination of training samples. The existing ship detection methods based on FVs often consider the Euclidean distances between target FVs and clutter FVs, where the subspace features of FVs are generally not exploited. In this article, we propose a new ship detection algorithm based on nonnegative sparse locality-representation (NSLR) to exploit the subspace features of FVs. The proposed NSLR method is based on the assumption that FVs of superpixels in SAR images are sparsely represented by the dictionary of background sea clutter only under a null hypothesis. In addition, we propose two FV-based filters to enhance the robustness of our newly developed NSLR to heterogeneous sea clutter environments by further exploiting the intrinsic features of ship targets in terms of intensity and spatiality. The experimental results based on Gaofen-3 SAR images demonstrate that the proposed NSLR detection method provides higher target-to-clutter contrast and achieves better detection performance than other commonly used ship detection algorithms.

An adaptive kernel sparse representation-based classification

In recent years, scholars have attached increasing attention to sparse representation. Based on compressed sensing and machine learning, sparse representation-based classification (SRC) has been extensively in classification. However, SRC is not suitable for samples with non-linear structures which arise in many practical applications. Meanwhile, sparsity is overemphasized by SRC, but the correlation information which is of great importance in classification is overlooked. To address these shortcomings, this study puts forward an adaptive kernel sparse representation-based classification (AKSRC). First, the samples were mapped to a high-dimensional feature space from the original feature space. Second, after selecting a suitable kernel function, a sample is represented as the linear combination of training samples of same class. Further more, the trace norm is adopted in AKSRC which is different from general approaches. It’s adaptive to the structure of dictionary which means that a better linear representation which has the most discriminative samples can be obtained. Therefore, AKSRC has more powerful classification ability. Finally, the advancement and effectiveness of the proposed AKSRC are verified by carrying out experiments on benchmark data sets.

Multi-variable estimation-based safe screening rule for small sphere and large margin support vector machine

Small Sphere and Large Margin (SSLM) SVM is one of the most competitive methods for Novelty Detection. However, the existing solvers for SSLM cannot deal with large data due to the expensive time cost. Although recently emerged safe screening methods can effectively enhance the computational speed, it is not available for SSLM because SSLM has multiple variables which cannot be represented explicitly by the linear combination of training samples. In this work, we construct a new safe screening rule for SSLM (MVE-SSR-SSLM) by integrating the ν-property, KKT conditions and variational inequalities. It is the first safe screening rule for a family of hypersphere support vector machine with multiple variables. The inactive samples are removed before actually solving the problem to accelerate the solving procedure without any loss of safety. Numerical experiments on fifteen benchmark datasets and Chinese wine dataset are conducted to show the validity and stability of the proposed MVE-SSR-SSLM.

Collaborative representation based classifier with partial least squares regression for the classification of spectral data

The need for effective methods to classify high-dimensional spectral data is increasing in tasks such as rapid and non-destructive detection of object features and chemical species using spectroscopy. Partial least squares discriminant analysis (PLS-DA) is an effective, multivariate regression based method for spectral data classification. Although powerful, PLS-DA suffers from performance degradation under complex conditions such as nonlinearity, class imbalance and multiclass, which are common in real-world applications. Collaborative representation-based classifier (CRC) is a new machine learning algorithm which represents a query by a linear combination of training samples and classifies the query based on the representation. It offers the possibility of good classification performance even under nonlinearity, class imbalance and multiclass conditions. In this paper, we present a novel method for spectral data classification, namely CRC-WPLS, which reaps the benefits of both PLS regression and CRC. This method searches for a weighted, linear combination of all training samples to represent the query by using PLS regression, and then assigns the query to the class which yields the least approximation error. CRC-WPLS is compared to PLS-DA, kernel PLS-DA, support vector machine (SVM), random forest (RF) and representation-based classifiers on fourteen general machine learning datasets and three spectral datasets. Experimental results show the proposed method can outperform 7 baseline methods in most cases, and achieve a high classification accuracy (>90%) for low grade spectra obtained from portable instrumentation.

Trimmed sparse coding for robust face recognition

In sparse representation (SR), a test image is encoded by a sparse linear combination of training samples. The L 1-regulariser used in SR is beneficial to produce a good reconstruction of the test face image with sparse error, but it is incapable to guarantee the robustness against local structural noise. To enhance noise tolerance of SR-based classifier, an improved L 1-regulariser based on trimmed sparse coding (TSC) by using an extra penalty on correlation among all coding coefficients is proposed. Different from traditional single-coding scheme in SR, multiple coding coefficients are used to represent patches of a test image by its corresponding training patches. The consistency penalty imposed into the new SR model improves the confidence for accuracy classification. Experimental results show the superiority of TSC on two benchmark databases, and it outperforms other state-of-the-art methods.

Sample partition and grouped sparse representation

Recently, conventional representation-based classification (RBC) methods demonstrate promising performance in image recognition. However, conventional RBCs only use a kind of deviations between the test sample and the linear combination of training samples of each class to perform classification. In many cases, a single kind of deviations corresponding to each class cannot effectively reflect the difference between the test sample and reconstructed sample of each class. Moreover, in practical applications, limited training samples are not able to reflect the possible changes of the image sufficiently. In this paper, we propose a novel scheme to tackle the above-mentioned problems. Specifically, we first use the original training samples to generate corresponding mirror samples. Thus, the original sample set and its mirror counterpart are treated as two separate training groups. Secondly, we perform collaborative representation classification on these two groups from which each class leads to two kinds of deviations, respectively. Finally, we fuse two kinds of deviations of each class and their correlation coefficient to classify the test sample. The correlation coefficient is defined for two kinds of deviations of each class. Experimental results on four databases show the proposed scheme can improve the recognition rate in image-based recognition.

Integrating Sparse and Collaborative Representation Classifications for Image Classification

Collaborative representation classification (CRC) is an important sparse method, which is easy to carry out and uses a linear combination of training samples to represent a test sample. CRC method utilizes the offset between representation result of each class and the test sample to implement classification. However, the offset usually cannot well express the difference between every class and the test sample. In this paper, we propose a novel representation method for image recognition to address the above problem. This method not only fuses sparse representation and CRC method to improve the accuracy of image recognition, but also has novel fusion mechanism to classify images. The implementations of the proposed method have the following steps. First of all, it produces collaborative representation of the test sample. That is, a linear combination of all the training samples is first determined to represent the test sample. Then, it gets the sparse representation classification (SRC) of the test sample. Finally, the proposed method respectively uses CRC and SRC representations to obtain two kinds of scores of the test sample and fuses them to recognize the image. The experiments of face recognition show that the combination of CRC and SRC has satisfactory performance for image classification.

Incorporating shape prior into active contours with a sparse linear combination of training shapes: application to corpus callosum segmentation.

In this paper, a novel method of embedding shape information into level set image segmentation is proposed. Our method is based on inferring shape variations by a sparse linear combination of instances in the shape repository. Given a sufficient number of training shapes with variations, a new shape can be approximated by a linear span of training shapes associated with those variations. At each step of curve evolution the curve is moved to minimize Chan-Vese energy functional as well as toward the best approximation based on a linear combination of training samples. Although the method is general, in this paper it has been applied to the problem of segmentation of corpus callosum from 2D sagittal MR images.

Kernel locality‐constrained sparse coding for head pose estimation

In many situations, it would be practical for a computer system user interface to have a model of where a person is looking and what the user is paying attention to. In this study, the authors describe a novel feature coding method for head pose estimation. The widely‐used sparse coding (SC) method encodes a test sample using a sparse linear combination of training samples. However, it does not consider the underlying structure of the data in the feature space. In contrast, locality‐constrained linear coding (LLC) utilises locality constraints to project each input data into its local‐coordinate system. Based on the recent success of LLC, the authors introduce locality‐constrained sparse coding (LSC) to overcome the limitation of Sparse Coding. The authors also propose kernel locality‐constrained sparse coding, which is a non‐linear extension of LSC. By using kernel tricks, the authors implicitly map the input data into the kernel feature space associated with the kernel function. In experiments, the proposed algorithm was applied to a head pose estimation application. Experimental results demonstrated the increased effectiveness and robustness of the method.

Robust structured sparse representation via half-quadratic optimization for face recognition

By representing a test sample with a linear combination of training samples, sparse representation-based classification (SRC) has shown promising performance in many applications such as computer vision and signal processing. However, there are several shortcomings in SRC such as 1) the l2-norm employed by SRC to measure the reconstruction fidelity is noise sensitive and 2) the l1-norm induced sparsity does not consider the correlation among the training samples. Furthermore, in real applications, face images with similar variations, such as illumination or expression, often have higher correlation than those from the same subject. Therefore, we correspondingly propose to improve the performance of SRC from two aspects by: 1) replacing the noise-sensitive l2-norm with an M-estimator to enhance its robustness and 2) emphasizing the sparsity in terms of the number of classes instead of the number of training samples, which leads to the structured sparsity. The formulated robust structured sparse representation (RGSR) model can be efficiently optimized via alternating minimization method under the half-quadratic (HQ) optimization framework. Extensive experiments on representative face data sets show that RGSR can achieve competitive performance in face recognition and outperforms several state-of-the-art methods in dealing with various types of noise such as corruption, occlusion and disguise.

Extended collaborative neighbor representation for robust single-sample face recognition

Appearance-based methods in face recognition have become one of the dominant techniques in recent years. Several studies have demonstrated that feature space is no longer critical in face recognition when the probe image can be approximated by a linear combination of training samples. However, in real-world application of face recognition, there are limited or even a single training sample per subject available. The availability of insufficient training samples often results in poor generalization ability for the existing state-of-the-art appearance-based methods. In this work, to address single-sample per person problem, we extend collaborative representation-based classification using $$l_{2}$$l2 minimization approach, named extended collaborative neighbor representation (ECNR) that exploits the representational ability of training samples and intrapersonal variation from generic subjects. The proposed method is based on the assumption that intrapersonal variations from generic subjects are sharable across training dictionary. ECNR expresses a test sample as the linear combination of optimal neighbor bases from the single class-specific training dictionary and the intrapersonal variant bases from an auxiliary generic subjects dictionary with multiple samples per subject to represent the possible variations in testing and training samples. The resulting coding scheme is sparse in terms of original bases and is computationally efficient. Experiments on real-world face data sets have demonstrated the usefulness of the proposed method in single-sample face recognition tasks.

Classification of hyperspectral urban data using adaptive simultaneous orthogonal matching pursuit

Simultaneous orthogonal matching pursuit (SOMP) has been recently developed for hyperspectral image classification. It utilizes a joint sparsity model with the assumption that each pixel can be represented by a linear combination of labeled samples. We present an approach to improve the performance of SOMP based on a priori segmentation map. According to the map, we first build a local region where within-segment pixels are preserved while between-segment pixels are excluded. Hyperspectral pixels in the preserved region around the test pixel are then simultaneously represented by a linear combination of training samples, whose weights are recovered by solving a sparsity-constrained optimization problem. Finally, the label of the test pixel is determined to be the class that yields the minimal total residuals between the test samples and the approximations. Experimental results demonstrate that the proposed adaptive SOMP (ASOMP) is superior to some existing classifiers, such as the original SOMP and the recently proposed weighted-SOMP (WSOMP). For example, the ASOMP performed with an accuracy of 95.53% for the ROSIS University of Pavia data with 120 training samples per class, while SOMP obtained an accuracy of 87.61%, an improvement of approximately 8%.

Nearest Regularized Subspace for Hyperspectral Classification

A classifier that couples nearest-subspace classification with a distance-weighted Tikhonov regularization is proposed for hyperspectral imagery. The resulting nearest-regularized-subspace classifier seeks an approximation of each testing sample via a linear combination of training samples within each class. The class label is then derived according to the class which best approximates the test sample. The distance-weighted Tikhonov regularization is then modified by measuring distance within a locality-preserving lower-dimensional subspace. Furthermore, a competitive process among the classes is proposed to simplify parameter tuning. Classification results for several hyperspectral image data sets demonstrate superior performance of the proposed approach when compared to other, more traditional classification techniques.

Kernel sparse representation based classification

Sparse representation has attracted great attention in the past few years. Sparse representation based classification (SRC) algorithm was developed and successfully used for classification. In this paper, a kernel sparse representation based classification (KSRC) algorithm is proposed. Samples are mapped into a high dimensional feature space first and then SRC is performed in this new feature space by utilizing kernel trick. Since samples in the high dimensional feature space are unknown, we cannot perform KSRC directly. In order to overcome this difficulty, we give the method to solve the problem of sparse representation in the high dimensional feature space. If an appropriate kernel is selected, in the high dimensional feature space, a test sample is probably represented as the linear combination of training samples of the same class more accurately. Therefore, KSRC has more powerful classification ability than SRC. Experiments of face recognition, palmprint recognition and finger-knuckle-print recognition demonstrate the effectiveness of KSRC.

A novel kernel-based framework for facial-image hallucination

In this paper, we present a kernel-based eigentransformation framework to hallucinate the high-resolution (HR) facial image of a low-resolution (LR) input. The eigentransformation method is a linear subspace approach, which represents an image as a linear combination of training samples. Consequently, those novel facial appearances not included in the training samples cannot be super-resolved properly. To solve this problem, we devise a kernel-based extension of the eigentransformation method, which takes higher-order statistics of the image data into account. To generate HR face images with higher fidelity, the HR face image reconstructed using this kernel-based eigentransformation method is treated as an initial estimation of the target HR face. The corresponding high-frequency components of this estimation are extracted to form a prior in the maximum a posteriori (MAP) formulation of the SR problem so as to derive the final reconstruction result. We have evaluated our proposed method using different kernels and configurations, and have compared these performances with some current SR algorithms. Experimental results show that our kernel-based framework, along with a proper kernel, can produce good HR facial images in terms of both visual quality and reconstruction errors.