A multi-average based pseudo nearest neighbor classifier

Improved pseudo nearest neighbor classification

In this paper, we propose a new reliable classification approach, called the pseudo nearest centroid neighbour rule, which is based on the pseudo nearest neighbour rule (PNN) and nearest centroid neighbourhood (NCN). In the proposed PNCN, the nearest centroid neighbours rather than nearest neighbours per class are first searched by means of NCN. Then, we calculate k categorical local mean vectors corresponding to k nearest centroid neighbours, and assign a weight to each local mean vector. Using the weighted k local mean vectors for each class, PNCN designs the corresponding pseudo nearest centroid neighbour and decides the class label of the query pattern according to the closest pseudo nearest centroid neighbour among all classes. The classification performance of the proposed PNCN is evaluated on real and artificial datasets in terms of the classification accuracy. The experimental results demonstrate the effectiveness and robustness of PNCN over the competing methods in many practical classification problems.

A generalized mean distance-based k-nearest neighbor classifier

A new k-harmonic nearest neighbor classifier based on the multi-local means

A new two-layer nearest neighbor selection method for kNN classifier

In this paper, we propose a new reliable classification approach, called the pseudo nearest centroid neighbor rule, which is based on the pseudo nearest neighbor rule (PNN) and nearest centroid neighborhood (NCN). In the proposed PNCN, the nearest centroid neighbors rather than nearest neighbors per class are first searched by means of NCN. Then, we calculate k categorical local mean vectors corresponding to k nearest centroid neighbors, and assign the weight to each local mean vector. Using the weighted k local mean vectors for each class, PNCN designs the corresponding pseudo nearest centroid neighbor and decides the class label of the query pattern according to the closest pseudo nearest centroid neighbor among all classes. The classification performance of the proposed PNCN is evaluated on real data sets in terms of the classification accuracy. The experimental results demonstrate the effectiveness of PNCN over the competing methods in many practical classification problems.

The well-known k-Nearest Neighbor classifier is a simple and flexible algorithm that has sparked wide interest in pattern classification. In spite of its straightforward implementation, the kNN is sensitive to the presence of noisy training samples and variance of the distribution. Local mean based k-nearest neighbor rule has been developed to overcome the negative effect of the noisy training sample. In this article, the local mean rule is implemented with the general nearest neighbors that are selected in a more generalized way. A new local mean based nearest neighbor classifier is proposed termed Local Mean k-General Nearest Neighbor (LMkGNN). The proposed LMkGNN classifier finds the local mean vector from general nearest neighbors of each class and classifies the test sample based on the distances between the test sample and local mean vectors. Fifteen real-world datasets from the UCI machine learning repository are used to assess and evaluate the classification performance of the proposed classifier. The performance comparison is also made with five benchmark classifiers (kNN, PNN, LMkNN, LMPNN and kGNN) in terms of the classification accuracy. Experimental results demonstrate that the proposed LMkGNN classifier performs significantly well and obtain the best classification accuracy com-pared to the five competing classifiers.

Attention-based Local Mean [formula omitted]-Nearest Centroid Neighbor Classifier

The k -nearest neighbor (KNN) rule is a successful technique in pattern classification due to its simplicity and effectiveness. As a supervised classifier, KNN classification performance usually suffers from low-quality samples in the training data set. Thus, training data set cleaning (TDC) methods are needed for enhancing the classification accuracy by cleaning out noisy, or even wrong, samples in the original training data set. In this paper, we propose a classification ability ranking (CAR)-based TDC method to improve the performance of a KNN classifier, namely CAR-based TDC method. The proposed classification ability function ranks a training sample in terms of its contribution to correctly classify other training samples as a KNN through the leave-one-out (LV1) strategy in the cleaning stage. The training sample that likely misclassifies the other samples during the KNN classifications according to the LV1 strategy is considered to have lower classification ability and will be cleaned out from the original training data set. Extensive experiments, based on ten real-world data sets, show that the proposed CAR-based TDC method can significantly reduce the classification error rates of KNN-based classifiers, while reducing computational complexity thanks to a smaller cleaned training data set.

The k-nearest neighbor (KNN) rule is a simple and effective nonparametric classification algorithm in pattern classification. However, it suffers from several problems such as sensitivity to outliers and inaccurate classification decision rule. Thus, a local mean-based k-nearest neighbor classifier (LMKNN) was proposed to address these problems, which assigns the query sample with a class label based on the closest local mean vector among all classes. It is proven that the LMKNN classifier achieves better classification performance and is more robust to outliers than the classical KNN classifier. Nonetheless, the unreliable nearest neighbor selection rule and single local mean vector strategy in LMKNN classifier severely have negative effect on its classification performance. Considering these problems in LMKNN, we propose a globally adaptive k-nearest neighbor classifier based on local mean optimization, which utilizes the globally adaptive nearest neighbor selection strategy and the implementation of local mean optimization to obtain more convincing and reliable local mean vectors. The corresponding experimental results conducted on twenty real-world datasets demonstrated that the proposed classifier achieves better classification performance and is less sensitive to the neighborhood size $$k$$ compared with other improved KNN-based classification methods.

Frog sound identification based on the vocalization becomes important for biological research and environmental monitoring. As a result, different types of feature extractions and classifiers have been employed to evaluate the accuracy of frog sound identification. This paper presents a frog sound identification with Extended k-Nearest Neighbor (EKNN) classifier. The EKNN classifier integrates the nearest neighbors and mutual sharing of neighborhood concepts, with the aims of improving the classification performance. It makes a prediction based on who are the nearest neighbors of the testing sample and who consider the testing sample as their nearest neighbors. In order to evaluate the classification performance in frog sound identification, the EKNN classifier is compared with competing classifier, k -Nearest Neighbor (KNN), Fuzzy k -Nearest Neighbor (FKNN) k - General Nearest Neighbor (KGNN)and Mutual k -Nearest Neighbor (MKNN) on the recorded sounds of 15 frog species obtained in Malaysia forest. The recorded sounds have been segmented using Short Time Energy and Short Time Average Zero Crossing Rate (STE+STAZCR), sinusoidal modeling (SM), manual and the combination of Energy (E) and Zero Crossing Rate (ZCR) (E+ZCR) while the features are extracted by Mel Frequency Cepstrum Coefficient (MFCC). The experimental results have shown that the EKNCN classifier exhibits the best performance in terms of accuracy compared to the competing classifiers, KNN, FKNN, GKNN and MKNN for all cases.

K-nearest neighbor (KNN) rule is a simple and effective algorithm in pattern classification. In this article, we propose a local mean-based k-nearest centroid neighbor classifier that assigns to each query pattern a class label with nearest local centroid mean vector so as to improve the classification performance. The proposed scheme not only takes into account the proximity and spatial distribution of k neighbors, but also utilizes the local mean vector of k neighbors from each class in making classification decision. In the proposed classifier, a local mean vector of k nearest centroid neighbors from each class for a query pattern is well positioned to sufficiently capture the class distribution information. In order to investigate the classification behavior of the proposed classifier, we conduct extensive experiments on the real and synthetic data sets in terms of the classification error. Experimental results demonstrate that our proposed method performs significantly well, particularly in the small sample size cases, compared with the state-of-the-art KNN-based algorithms.

Learning label noise is gaining increasing attention from a variety of disciplines, particularly in supervised machine learning for classification tasks. The k nearest neighbors (kNN) classifier is often used as a natural way to edit the training sets due to its sensitivity to label noise. However, the kNN-based editor may remove too many instances if not designed to take care of the label noise. In addition, the one-sided nearest neighbor (NN) rule is unconvincing, as it just considers the nearest neighbors from the perspective of the query sample. In this paper, we propose an ensemble and iterative recovery strategy-based kGNN method (EIRS-kGNN) to edit data with label noise. EIRS-kGNN first uses the general nearest neighbors (GNN) to expand the one-sided NN rule to a binary-sided NN rule, taking the neighborhood of the queried samples into account. Then, it ensembles the prediction results of a finite set of ks in the kGNN to prudently judge the noise levels for each sample. Finally, two loops, i.e., the inner loop and the outer loop, are leveraged to iteratively detect label noise. A frequency indicator is derived from the iterative processes to guide the mixture approaches, including relabeling and removing, to deal with the detected label noise. The goal of EIRS-kGNN is to recover the distribution of the data set as if it were not corrupted. Experimental results on both synthetic data sets and UCI benchmarks, including binary data sets and multi-class data sets, demonstrate the effectiveness of the proposed EIRS-kGNN method.

Industrial data usually have nonlinear or multimodal characteristics which do not meet the data assumptions of statistics in principal component analysis (PCA). Therefore, PCA has a lower fault detection rate in industrial processes. Aiming at the above limitations of PCA, a fault detection method using principal component difference based on k-nearest neighbors (Diff-PCA) is proposed in this paper. First, find the k nearest neighbors set of each sample in the training data set and calculate its mean vector. Second, build an augmented vector using each sample and its corresponding mean vector. Third, calculate the loading matrix and score matrix using PCA. Next, calculate the estimated scores using the mean vector of each sample and missing data imputation technique for PCA. At last, build two new statistics using the difference between the real scores and estimated scores to detect faults. In addition, the fault diagnosis method based on contribution plots of monitored variables is also proposed in this paper. In Diff-PCA, the difference skill can eliminate the impact of the nonlinear and multimodal structure on fault detection. Meanwhile, the monitored subspaces by the two new statistics are different from that by T2 and SPE in PCA. The efficiency of the proposed strategy is implemented in two numerical cases (nonlinear and multimode) and the Tennessee Eastman (TE) processes. The fault detection results indicate that Diff-PCA outperforms the conventional PCA, Kernel PCA, dynamic PCA, principal component-based k nearest neighbor rule and k nearest neighbor rule.

K-nearest neighbor (KNN) rule is a simple and effective classifier in pattern recognition. In this paper, we propose a new nearest neighbor classifier based on multi-harmonic mean distances, in order to overcome the sensitivity of the neighborhood size k and improve the classification performance. The proposed method is called a harmonic mean distance-based k-nearest neighbor classifier (HMDKNN). It mainly designs the multi-harmonic mean distances based on the multi-local mean vectors calculated by utilizing k nearest neighbors of the given query sample in each class. Using the multi-harmonic mean distances per class, a new nested harmonic mean distance in each class is designed as the classification decision and the query sample is classified into the class with the closest nested harmonic mean distance among all classes. The experimental results on the UCI data sets show that the proposed HMDKNN performs better with the less sensitiveness to k, compared to the state-of-art KNN-based methods.