Condensed Nearest Neighbor Research Articles

A Comparative Survey of Instance Selection Methods applied to Non-Neural and Transformer-Based Text Classification

Progress in natural language processing has been dictated by the rule of more : more data, more computing power, more complexity, best exemplified by deep learning Transformers. However, training (or fine-tuning) large dense models for specific applications usually requires significant amounts of computing resources. One way to ameliorate this problem is through data engineering instead of the algorithmic or hardware perspectives. Our focus here is an under-investigated data engineering technique, with enormous potential in the current scenario – Instance Selection (IS) (a.k.a. Selective Sampling, Prototype Selection). The IS goal is to reduce the training set size by removing noisy or redundant instances while maintaining or improving the effectiveness (accuracy) of the trained models and reducing the training process cost. We survey classical and recent state-of-the-art IS techniques and provide a scientifically sound comparison of IS methods applied to an essential natural language processing task—Automatic Text Classification (ATC). IS methods have been normally applied to small tabular datasets and have not been systematically compared in ATC. We consider several neural and non-neural state-of-the-art ATC solutions and many datasets. We answer several research questions based on tradeoffs induced by a tripod (training set reduction, effectiveness, and efficiency). Our answers reveal an enormous unfulfilled potential for IS solutions. Specially, we show that in 12 out of 19 datasets, specific IS methods—namely, Condensed Nearest Neighbor, Local Set-based Smoother, and Local Set Border Selector—can reduce the size of the training set without effectiveness losses. Furthermore, in the case of fine-tuning the Transformer methods, the IS methods reduce the amount of data needed, without losing effectiveness and with considerable training-time gains.

Individual Online Learning Behavior Analysis Based on Hadoop.

The online individual behavior analysis is an important means for mining user interests. The user retweeting behavior prediction is typical problem for online individual behavior analysis. In order to make online learning behavior prediction method more suitable for the application of large-scale datasets, the improved condensed K nearest neighbor (ICKNN) method is proposed in this paper. Inspired by the idea of compressing samples in the condensed nearest neighbor (CNN) algorithm, this proposed method has adopted the Hadoop platform to parallelize the traditional CNN algorithm. For the traditional CNN method, as the value of K increases, the compression ratio decreases and so as the efficiency. The proposed ICKNN method can parallelize the traditional CNN method under the Hadoop framework to enhance efficiency. The proposed ICKNN method in this paper is validated by actual Twitter retweeting dataset. It can be seen that the proposed method in this paper has a higher compression rate than the traditional CNN algorithm. In terms of accuracy, the classification accuracy of the proposed ICKNN method has decreased compared with the traditional KNN method. However, the time consumed by the ICKNN method has significantly reduced compared with the traditional KNN method and CNN method, which can greatly improve the efficiency.

An independent condensed nearest neighbor classification technique for medical image retrieval

Nowadays, medical images play a vital role in the clinical diagnosis system, because these images contain a vast amount of medical information. The huge amount of medical images is generated everyday using the digital imaging techniques in medical examination centers and hospitals. The generated data are stored in large database and retrieving the same images is a significant task for better diagnosis. However, manual retrieval of clinical data is a tough and time consuming process due to the large database. The memory requirement and computation complexity of the existing condensed nearest neighbor (CNN) is more, it is solved by proposed independent condensed nearest neighbor (ICNN). The ICNN classification technique is developed to automatically retrieve the medical images from large database. The important features are extracted by using histogram of gradient (HOG) technique. The sub-quadratic time complexity presented in ICNN requires only few iterations to retrieve the query images, which improves the retrieval accuracy of the proposed technique. The experiments are used to test the performance of ICNN method in terms of classification and retrieval presentation. The proposed ICNN method outperforms existing CNN method by achieving specificity of 99.55% in the classification performance.

A Rebalancing Framework for Classification of Imbalanced Medical Appointment No-show Data

Abstract Purpose This paper aims to improve the classification performance when the data is imbalanced by applying different sampling techniques available in Machine Learning. Design/methodology/approach The medical appointment no-show dataset is imbalanced, and when classification algorithms are applied directly to the dataset, it is biased towards the majority class, ignoring the minority class. To avoid this issue, multiple sampling techniques such as Random Over Sampling (ROS), Random Under Sampling (RUS), Synthetic Minority Oversampling TEchnique (SMOTE), ADAptive SYNthetic Sampling (ADASYN), Edited Nearest Neighbor (ENN), and Condensed Nearest Neighbor (CNN) are applied in order to make the dataset balanced. The performance is assessed by the Decision Tree classifier with the listed sampling techniques and the best performance is identified. Findings This study focuses on the comparison of the performance metrics of various sampling methods widely used. It is revealed that, compared to other techniques, the Recall is high when ENN is applied CNN and ADASYN have performed equally well on the Imbalanced data. Research limitations The testing was carried out with limited dataset and needs to be tested with a larger dataset. Practical implications This framework will be useful whenever the data is imbalanced in real world scenarios, which ultimately improves the performance. Originality/value This paper uses the rebalancing framework on medical appointment no-show dataset to predict the no-shows and removes the bias towards minority class.

An instance selection algorithm for fuzzy K-nearest neighbor

The condensed nearest neighbor (CNN) is a pioneering instance selection algorithm for 1-nearest neighbor. Many variants of CNN for K-nearest neighbor have been proposed by different researchers. However, few studies were conducted on condensed fuzzy K-nearest neighbor. In this paper, we present a condensed fuzzy K-nearest neighbor (CFKNN) algorithm that starts from an initial instance set S and iteratively selects informative instances from training set T, moving them from T to S. Specifically, CFKNN consists of three steps. First, for each instance x ∈ T, it finds the K-nearest neighbors in S and calculates the fuzzy membership degrees of the K nearest neighbors using S rather than T. Second it computes the fuzzy membership degrees of x using the fuzzy K-nearest neighbor algorithm. Finally, it calculates the information entropy of x and selects an instance according to the calculated value. Extensive experiments on 11 datasets are conducted to compare CFKNN with four state-of-the-art algorithms (CNN, edited nearest neighbor (ENN), Tomeklinks, and OneSidedSelection) regarding the number of selected instances, the testing accuracy, and the compression ratio. The experimental results show that CFKNN provides excellent performance and outperforms the other four algorithms.

A machine learning NOxemission model for SCR system considering mechanism knowledge and catalyst deactivation

In this work, an adaptive NOxemission model is proposed for a SCR system of a 660 MW utility boiler. First, 3-years operating data was collected from the plant SIS system as raw data, which was then filtered using the R-statistic method and clustered by the condensed nearest neighbor (CNN) rule to form a classified steady-state database. In addition, a sliding window approach was used to deal with the continuous data stream. As the newest steady state sample was introduced into the database, the most similar old sample in the same data class was replaced. The crowding distance (CD) operator was also used to eliminate the redundant samples. This new method RCNN-CD is proven to be a good tool to improve the representatives of the samples. Based on the selected samples, a fusion monotony support vector regression (FM-SVR) was used to establish the NOxemission model. The results show that, this model can reasonably reflect SCR mechanism and follow the degradation of SCR performance.

Design and Evaluation of Outlier Detection Based on Semantic Condensed Nearest Neighbor

Abstract Social media contain abundant information about the events or news occurring all over the world. Social media growth has a greater impact on various domains like marketing, e-commerce, health care, e-governance, and politics, etc. Currently, Twitter was developed as one of the social media platforms, and now, it is one of the most popular social media platforms. There are 1 billion user’s profiles and millions of active users, who post tweets daily. In this research, buzz detection in social media was carried out by the semantic approach using the condensed nearest neighbor (SACNN). The Twitter and Tom’s Hardware data are stored in the UC Irvine Machine Learning Repository, and this dataset is used in this research for outlier detection. The min–max normalization technique is applied to the social media dataset, and additionally, missing values were replaced by the normalized value. The condensed nearest neighbor (CNN) is used for semantic analysis of the database, and based on the optimized value provided by the proposed method, the threshold is calculated. The threshold value is used to classify buzz and non-buzz discussions in the social media database. The result showed that the SACNN achieved 99% of accuracy, and relative error is less than the existing methods.

Redundancy-driven modified Tomek-link based undersampling: A solution to class imbalance

Class imbalance can be defined as a span among data mining, machine learning and pattern recognition domains that provides to learn from a data-space having unequal class distribution. Common classifiers when trained by imbalanced data tend to bias towards the class possessing bulk instances causing misclassification of upcoming patterns/instances. The study reveals that presence of redundant borderline instances and outliers in the data-space severely catalyzes the effect of class imbalance. The Condensed Nearest Neighbor and Tomek-link undersampling techniques are used as the baseline systems for the present study, and an improved undersampling algorithm is proposed to be employed in the pre-processing stage by amalgamating aspects of outlier and redundancy detection to the baseline system. The proposed scheme imparts to detect outlier, redundant and noisy instances having least contribution in estimating accurate class labels. Thus, a data-level solution has been offered to the concerned problem with novelty in effective elimination of majority instances without losing valuable information. The proposed scheme is implemented and validated with Back Propagation Neural Network (BPNN), K-Nearest-Neighbor (K-NN), Support Vector Machine (SVM) and Naive Bayes classifiers for 10 real-life datasets. The experimental results obtained clearly manifest the superiority of the proposed scheme over the baseline schemes.

Fusion of instance selection methods in regression tasks

Data pre-processing is a very important aspect of data mining. In this paper we discuss instance selection used for prediction algorithms, which is one of the pre-processing approaches. The purpose of instance selection is to improve the data quality by data size reduction and noise elimination. Until recently, instance selection has been applied mainly to classification problems. Very few recent papers address instance selection for regression tasks. This paper proposes fusion of instance selection algorithms for regression tasks to improve the selection performance. As the members of the ensemble two different families of instance selection methods are evaluated: one based on distance threshold and the other one on converting the regression task into a multiple class classification task. Extensive experimental evaluation performed on the two regression versions of the Edited Nearest Neighbor (ENN) and Condensed Nearest Neighbor (CNN) methods showed that the best performance measured by the error value and data size reduction are in most cases obtained for the ensemble methods.

Condensed Nearest Neighbor Decision Rule Input Weight Sequential Feed-Forward Neural Networks

This paper proposes a new approach is referred to as condensed nearest neighbor decision rule (CNN) input weight sequential feed-forward neural networks (CIW-SFFNS). In this paper, it is firstly shown that the difference of optimization constraints between the extreme learning machine (ELM) and constrained-optimization-based extreme learning machine. For the second time, this paper proposes a method that using CNN to select the hidden-layer weights from example. Moreover, we compare error minimized extreme learning machines (EM-ELM), support vector sequential feed-forward neural networks (SV-SFFNS) and CIW-SFFNS from two aspects:test accuracy and the number of hidden nodes. We present the result of an experimental study on 10 classification sets. The CIW-SFFNS algorithm has a statistically significant improvement in generalization performance than EM-ELM and SV-SFFNS.

Ensembles of Instance Selection Methods based on Feature Subset

In this paper the application of ensembles of instance selection algorithms to improve the quality of dataset size reduction is evaluated. In order to ensure diversity of sub models, selection of a feature subsets was considered. In the experiments the Condensed Nearest Neighbor (CNN) and Edited Nearest Neighbor (ENN) algorithms were evaluated as basic instance selection methods. The results show that it is possible to obtain various trade-offs between data compression and classification accuracy depending on the acceptance threshold and feature ratio parameters. In some cases it was possible to achieve both: higher compression and higher accuracy than those of an individual instance selection algorithm.

The condensed nearest neighbor rule using the concept of mutual nearest neighborhood (Corresp.)

A two-stage iterative algorithm for selecting a subset of a training set of samples for use in a condensed nearest neighbor (CNN) decision rule is introduced. The proposed method uses the concept of mutual nearest neighborhood for selecting samples close to the decision line. The efficacy of the algorithm is brought out by means of an example.