Local means-based fuzzy k-nearest neighbor classifier with Minkowski distance and relevance-complementarity feature weighting

A new fuzzy k-nearest neighbor classifier based on the Bonferroni mean

The k-nearest neighbor (KNN) rule is a simple and effective nonparametric classification algorithm in pattern classification. However, it suffers from several problems such as sensitivity to outliers and inaccurate classification decision rule. Thus, a local mean-based k-nearest neighbor classifier (LMKNN) was proposed to address these problems, which assigns the query sample with a class label based on the closest local mean vector among all classes. It is proven that the LMKNN classifier achieves better classification performance and is more robust to outliers than the classical KNN classifier. Nonetheless, the unreliable nearest neighbor selection rule and single local mean vector strategy in LMKNN classifier severely have negative effect on its classification performance. Considering these problems in LMKNN, we propose a globally adaptive k-nearest neighbor classifier based on local mean optimization, which utilizes the globally adaptive nearest neighbor selection strategy and the implementation of local mean optimization to obtain more convincing and reliable local mean vectors. The corresponding experimental results conducted on twenty real-world datasets demonstrated that the proposed classifier achieves better classification performance and is less sensitive to the neighborhood size $$k$$ compared with other improved KNN-based classification methods.

This paper reports on advances in identification of relevant features through iterative feature weighting with radial basis function networks. It proceeds with a set of feature weights to scale the data which are used to train a radial basis function network model. Then from the learned model, the feature weights are updated via one-step gradient descent. The updated feature weights are then fed back to build a new model. The procedure continues until we find a satisfactory model and the feature weights converge. Experimental results for some benchmark datasets show that the approach is efficient and effective for selecting relevant features for data modeling and classification tasks.

In feature weighted fuzzy c-means algorithms, there exist two challenges when the feature weighting techniques are used to improve their performances. On one hand, if the values of feature weights are learnt in advance, and then fixed in the process of clustering, the learnt weights might be lack of flexibility and might not fully reflect their relevance. On the other hand, if the feature weights are adaptively adjusted during the clustering process, the algorithms maybe suffer from bad initialization and lead to incorrect feature weight assignment, thus the performance of the algorithms may degrade the in some conditions. In order to ease these problems, a novel weighted fuzzy c-means based on feature weight learning (FWL-FWCM) is proposed. It is a hybrid of fuzzy weighted c-means (FWCM) algorithm with Improved FWCM (IFWCM) algorithm. FWL-FWCM algorithm first learns feature weights as priori knowledge from the data in advance by minimizing the feature evaluation function using the gradient descent technique, then iteratively optimizes the clustering objective function which integrates the within weighted cluster dispersion with a term of the discrepancy between the weights and the priori knowledge. Experiments conducted on an artificial dataset and real datasets demonstrate the proposed approach outperforms the-state-of-the-art feature weight clustering methods. The convergence property of FWL-FWCM is also presented.

K-nearest neighbor (KNN) rule is a simple and effective algorithm in pattern classification. In this article, we propose a local mean-based k-nearest centroid neighbor classifier that assigns to each query pattern a class label with nearest local centroid mean vector so as to improve the classification performance. The proposed scheme not only takes into account the proximity and spatial distribution of k neighbors, but also utilizes the local mean vector of k neighbors from each class in making classification decision. In the proposed classifier, a local mean vector of k nearest centroid neighbors from each class for a query pattern is well positioned to sufficiently capture the class distribution information. In order to investigate the classification behavior of the proposed classifier, we conduct extensive experiments on the real and synthetic data sets in terms of the classification error. Experimental results demonstrate that our proposed method performs significantly well, particularly in the small sample size cases, compared with the state-of-the-art KNN-based algorithms.

Investigating the impact of data normalization on classification performance

Many feature weighting methods have been proposed to evaluate feature saliencies in recent years. Neural-network (NN) feature weighting, as a supervised method, is founded upon the mapping from input features to output decisions, and implemented by evaluating the sensitivity of network outputs to its inputs. Through training on sample data, NN implicitly embodies the saliencies of input features. The partial derivatives of the outputs with respect to the inputs in the trained NN are calculated to measure their sensitivities to input features, which means that implicit feature weighting of the NN is transformed into explicit feature weighting. The purpose of this paper is to further probe into the principle of NN feature weighting, and evaluate its performance through a comparative study between NN feature weighting method and state-of-art weighting methods in the same working conditions. The motivation of this study is inspired by the lack of direct and comprehensive comparison studies of NN feature weighting method. Experiments in UCI repository data sets, face data sets and self-built data sets show that NN feature weighting method achieves superior performance in different conditions and has promising prospects. Compared with the other existing methods, NN feature weighting method can be used in more complex conditions, provided that NN can work in those conditions. As decision data, output data can be labels, reals or integers. Especially, feature weights can be calculated without the discretization of outputs in the condition of continuous outputs.

Feature weighting is used to alleviate the conditional independence assumption of Naïive Bayes text classifiers and consequently improve their generalization performance. Most traditional feature weighting algorithms use general feature weighting, which assigns the same weight to each feature for all classes. We focus on class-specific feature weighting approaches, which discriminatively assign each feature a specific weight for each class. This paper uses a statistical feature weighting technique and proposes a new class-specific deep feature weighting method for Multinomial Naïve Bayes text classifiers. In this deep feature weighting method, feature weights are not only incorporated into the classification formulas but they are also incorporated into the conditional probability estimates of Multinomial Naïve Bayes text classifiers. Experimental results for a large number of text classification datasets validate the effectiveness and efficiency of our method.

Deep feature weighting for naive Bayes and its application to text classification

In this paper, Feature weighting is used to create an intelligent and effective classification method for Kinnow fruits. Feature weighting approach is used because it improves classification performance more than feature selection methods. The modified sunflower optimization algorithm (SFO) is proposed to search the optimal feature weights and parametric values of k-Nearest Neighbour (kNN).The levy flight distribution operator has been utilised to enhance the convergence speed of the sunflower optimization algorithm by improving the local and global search ability of the optimization algorithm. Also, the algorithmic parameter of the SFO algorithm has been adaptively selected using the linear time varying adaption method. In addition, tanh normalization technique is used for the data pre-processing to reduce the influence of outliers and dominating features before the feature weighting method. The findings suggest that the proposed wrapper based approach feature weighting technique is more capable of achieving higher accuracy than the existing strategies.

Safe dynamic sparse training of modified RBF networks for joint feature selection and classification

In a real-world data set, there is always the possibility, rather high in our opinion, that different features may have different degrees of relevance. Most machine learning algorithms deal with this fact by either selecting or deselecting features in the data preprocessing phase. However, we maintain that even among relevant features there may be different degrees of relevance, and this should be taken into account during the clustering process. With over 50 years of history, K-Means is arguably the most popular partitional clustering algorithm there is. The first K-Means based clustering algorithm to compute feature weights was designed just over 30 years ago. Various such algorithms have been designed since but there has not been, to our knowledge, a survey integrating empirical evidence of cluster recovery ability, common flaws, and possible directions for future research. This paper elaborates on the concept of feature weighting and addresses these issues by critically analyzing some of the most popular, or innovative, feature weighting mechanisms based in K-Means.

In a real-world data set there is always the possibility, rather high in our opinion, that different features may have different degrees of relevance. Most machine learning algorithms deal with this fact by either selecting or deselecting features in the data preprocessing phase. However, we maintain that even among relevant features there may be different degrees of relevance, and this should be taken into account during the clustering process. With over 50 years of history, K-Means is arguably the most popular partitional clustering algorithm there is. The first K-Means based clustering algorithm to compute feature weights was designed just over 30 years ago. Various such algorithms have been designed since but there has not been, to our knowledge, a survey integrating empirical evidence of cluster recovery ability, common flaws, and possible directions for future research. This paper elaborates on the concept of feature weighting and addresses these issues by critically analysing some of the most popular, or innovative, feature weighting mechanisms based in K-Means

Simultaneous feature weighting and parameter determination of Neural Networks using Ant Lion Optimization for the classification of breast cancer

Improved pseudo nearest neighbor classification