Remaining Training Samples Research Articles

KADEL: Knowledge-Aware Denoising Learning for Commit Message Generation

Commit messages are natural language descriptions of code changes, which are important for software evolution such as code understanding and maintenance. However, previous methods are trained on the entire dataset without considering the fact that a portion of commit messages adhere to good practice (i.e., good-practice commits), while the rest do not. On the basis of our empirical study, we discover that training on good-practice commits significantly contributes to the commit message generation. Motivated by this finding, we propose a novel knowledge-aware denoising learning method called KADEL. Considering that good-practice commits constitute only a small proportion of the dataset, we align the remaining training samples with these good-practice commits. To achieve this, we propose a model that learns the commit knowledge by training on good-practice commits. This knowledge model enables supplementing more information for training samples that do not conform to good practice. However, since the supplementary information may contain noise or prediction errors, we propose a dynamic denoising training method. This method composes a distribution-aware confidence function and a dynamic distribution list, which enhances the effectiveness of the training process. Experimental results on the whole MCMD dataset demonstrate that our method overall achieves state-of-the-art performance compared with previous methods.

Robust and optimal epsilon-insensitive Kernel-based regression for general noise models

Sparse representation of kernel based regression (KBR) has received considerable attention in recent years. Studies on sparse KBR can be divided into two distinct groups, namely (i) pruning-based methods that remove the training samples with the least training errors and retrain the remaining training samples, and (ii) direct methods that begin with a full-dense solution and delete training data according to objective criteria. Pruning-based methods give rise to a high computation time, while direct methods may lead to non-optimal solutions and thus a poor approximation. In addition, most current KBR models assume that the error distribution is Gaussian. However, observations in many practical applications indicate that the noise models do not satisfy the Gaussian error distribution. In such cases, current KBR models are not optimal. To address the above-mentioned problems, this study proposes a new sparse KBR framework for general noise distributions, including the epsilon-insensitive noise family. Compared with other sparse algorithms, sparsity is directly imposed by epsilon-insensitive convex loss functions derived from the theoretical framework of the Bayesian approach within the scope of regularization networks, and then handles the optimization problem in Lagrangian form. Experiments on artificial and real-life benchmark datasets demonstrate that the proposed epsilon-insensitive KBR models are more effective and efficient than pruning-based approaches.

Cyclic training sample selection and cancellation technique for airborne STAP radar under nonhomogeneous environment

When training samples contain interference target signals (i.e., outliers), the performance of space–time adaptive processing (STAP) will be degraded. To solve this problem, a cyclic training sample selection and cancellation (CTSSC) algorithm based on sample matrix inversion (SMI) statistic is proposed. This algorithm consists of three steps. Firstly, a cyclic process is carried out to detect outliers. In each detection cycle, the sample matrix is estimated by all the training samples contained in the training sample set, and the outliers with relatively high jamming-to-noise ratio (JNR) will be detected. Then the corresponding training samples are removed from the training sample set and the sample matrix is updated. This cycle continues until the SMI statistics of all the remaining training samples do not exceed the detection threshold. Secondly, for each outlier detected by the first step, it will be reconstructed by the estimated complex amplitude and steering vector. Then the outlier will be cancelled by the corresponding reconstructed outlier. Finally, conventional STAP will be carried out to suppress clutter. Simulation results validate the effectiveness of the CTSSC algorithm with measured data.

Collaborative representation‐based locality preserving projections for image classification

Graph embedding has attracted much more research interests in dimensionality reduction. In this study, based on collaborative representation and graph embedding, the authors propose a new linear dimensionality reduction method called collaborative representation-based locality preserving projection (CRLPP). In the CRLPP, they assume that the similar samples should have similar reconstructions by collaborative representation and the similar reconstructions should also have the similar low-dimensional representations in the projected subspace. CRLPP first reconstructs each training sample using the collaborative representation of the other remaining training samples, and then designs the graph construction of all training samples, finally establishes the objective function of graph embedding using the collaborative reconstructions and the constructed graph. The proposed CRLPP can well preserve the intrinsic geometrical and discriminant structures of high-dimensional data in low-dimensional subspace. The effectiveness of the proposed is verified on several image datasets. The experimental results show that the proposed method outperforms the state-of-art dimensionality reduction.

Ramp Loss based robust one-class SVM

One-class SVM (OCSVM) is widely adopted in one-class classification (OCC) fields. However, outliers in the training set negatively influence the classification surface of OCSVM, degrading its performance. To solve this problem, a novel method is proposed in this paper. This proposed method introduces Ramp Loss function into OCSVM optimization, so as to reduce outliers’ influence. Then the outliers are identified and removed from the training set. The final classification surface is obtained on the remaining training samples. Various experiments verify the effectiveness of this proposed method.

Progressive sparse representation-based classification using local discrete cosine transform evaluation for image recognition

This paper proposes a progressive sparse representation-based classification algorithm using local discrete cosine transform (DCT) evaluation to perform face recognition. Specifically, the sum of the contributions of all training samples of each subject is first taken as the contribution of this subject, then the redundant subject with the smallest contribution to the test sample is iteratively eliminated. Second, the progressive method aims at representing the test sample as a linear combination of all the remaining training samples, by which the representation capability of each training sample is exploited to determine the optimal “nearest neighbors” for the test sample. Third, the transformed DCT evaluation is constructed to measure the similarity between the test sample and each local training sample using cosine distance metrics in the DCT domain. The final goal of the proposed method is to determine an optimal weighted sum of nearest neighbors that are obtained under the local correlative degree evaluation, which is approximately equal to the test sample, and we can use this weighted linear combination to perform robust classification. Experimental results conducted on the ORL database of faces (created by the Olivetti Research Laboratory in Cambridge), the FERET face database (managed by the Defense Advanced Research Projects Agency and the National Institute of Standards and Technology), AR face database (created by Aleix Martinez and Robert Benavente in the Computer Vision Center at U.A.B), and USPS handwritten digit database (gathered at the Center of Excellence in Document Analysis and Recognition at SUNY Buffalo) demonstrate the effectiveness of the proposed method.

A novel SRC fusion method using hierarchical multi-scale LBP and greedy search strategy

As is well known, most of the sparse representation based classification (SRC) schemes are not the standard form of SR model, that is, the dictionary in SRC model is not a strict one compare to sparse representation model. Actually, a strict and standard dictionary will create sparser coefficient, but it might not achieve better classification accuracy in SRC model. Therefore, constructing a rational and optimal dictionary for SRC model is a challenging task for us. Understanding the good performance of such unconventional dictionaries demands new algorithmic and analytical techniques. In this paper, we propose a novel SRC fusion method using hierarchical multi-scale local binary patterns (LBP) and greedy search strategy (SRC-GSLBP) for face recognition. The proposed method involves three aspects: dictionary optimization, training variables selection and classification strategy, sparse coding coefficient decomposing. First of all, we get an optimized dictionary through extracting hierarchical multi-scale LBP features from the original training samples. Second, the training variables selection and classification strategy aim to represent a query sample as a linear combination of the most informative training samples, and exploits an optimal representation of training samples from the classes with major relevant contributions. Instead of eliminating several classes at one time, we choose eliminating classes one by one with greedy search (GS) sparse coding process until the predefined termination condition is satisfied. The final remaining training samples are used to produce a best representation of the test sample and to perform classification. In the context of the proposed method, an important goal is to select a subset of variables that accomplishes one objective: the provision of a descriptive representation for sparse category knowledge structure. We develop a heuristic learning strategy to achieve this goal. Experimental results conducted on the ORL, FERET and AR face databases demonstrate the effectiveness of the proposed method.

Using idea of three-step sparse residuals measurement to perform discriminant analysis

Classification of high-dimensional data is usually not amenable to standard pattern recognition techniques owing to lack of necessary, underlying structured information of data. In this paper, we propose a new discriminant analysis based on three-step sparse residuals measurement called DA-TSSR to address this problem. Specifically, in the first stage of the proposed method, the contribution in presenting the test sample of any chosen class is respectively calculated by adding up the total contributions of all the training samples of this class, and then a certain class with the smallest contribution score is eliminated from the set of the training samples. This procedure is iteratively carried out for the set of the training samples of the remaining classes till the predefined termination condition is satisfied. The second stage of DA-TSSR seeks to represent the test sample as a linear combination of all the remaining training samples and exploits the representation ability of each training sample to determine M "nearest neighbors" for the test sample. By this means, it generates unequal number of training samples on each candidate class. The third stage of DA-TSSR again determines a new weighted sum of all unequal numbers of training samples from candidate classes, which is approximately equal to the test sample. We use the new weighted sum to perform the designing of sparse residuals grades, which can be incorporated into the typical discriminant analysis criterion. The proposed method not only has a high accuracy but also can be clearly interpreted. Experimental results conducted on the ORL, XM2VTS, FERET and AR face databases demonstrate the effectiveness of the proposed method.

A new sparse representation-based classification algorithm using iterative class elimination

In this paper, we propose a sparse representation-based classification method using iterative class elimination strategy for face recognition. The proposed method aims to represent a test sample as a linear combination of the most competitive training samples and exploits an optimal representation of training samples from the classes with major relevant contributions. We interpret the sparse representation problem as an information fidelity problem. In the context of our proposed method, an important goal is to select a subset of variables for accomplishing one objective: the provision of a descriptive representation for sparse class knowledge structure. We develop an iterative class elimination algorithm to achieve this goal. First, the contribution in presenting the test sample of any of the specified classes is, respectively, calculated by adding up the total contribution of all training samples of this class, and then a certain class that meets the smallest score requirement to this test sample is eliminated. Second, a similar procedure is iteratively carried out for the set of remaining training samples from rest classes, and this procedure is repeatedly performed till the predefined termination condition is satisfied. The final remaining training samples are used to produce a best representation of the test sample and to classify it. Therefore, the proposed algorithm is an iterative method that alternates between sparse representation and a process of updating the training atoms to better fit the test data. This is helpful to accurately classify the test sample. Experimental results conducted on the ORL, FERET, and AR face databases demonstrate the effectiveness of the proposed method.

Supervised sparse representation method with a heuristic strategy and face recognition experiments

In this paper we propose a supervised sparse representation method for face recognition. We assume that the test sample could be approximately represented by a sparse linear combination of all the training samples, where the term “ sparse” means that in the linear combination most training samples have zero coefficients. We exploit a heuristic strategy to achieve this goal. First, we determine a linear combination of all the training samples that best represents the test sample and delete the training sample whose coefficient has the minimum absolute value. Then a similar procedure is carried out for the remaining training samples and this procedure is repeatedly carried out till the predefined termination condition is satisfied. The finally remaining training samples are used to produce a best representation of the test sample and to classify it. The face recognition experiments show that the proposed method can achieve promising classification accuracy.