Instance-based Classification Research Articles

Multiple instance learning for medical image classification based on instance importance

Multiple Instance Learning (MIL) is designed to classify instances where class labels are associated with sets of instances, a common occurrence in biomedical data, especially when multiple images are derived from a single object measurement. Among the various MIL techniques, instance-based classification emerges as the main approach. Individual instances are assigned class labels, and the final bag-level classification is determined through an aggregation process, usually based on average or maximum values. While instance-based models may exhibit slightly lower performance compared to more general methods, they offer the unique advantage of interpretability, allowing the decision-making process to be traced to each individual instance. This article presents an innovative instance-based approach that not only assigns class labels to instances, but also gives them meaning. The final aggregation uses a weighted average, allowing the model to identify instances carrying valuable information while distinguishing those that are redundant in the classification process. This enhancement gives the model the ability to recognize the meaning of individual instances, thereby increasing both its overall interpretability and classification performance.

Nano scale instance-based learning using non-specific hybridization of DNA sequences

DNA, or deoxyribonucleic acid, is a powerful molecule that plays a fundamental role in storing and processing genetic information of all living organisms. In recent years, scientists have harnessed hybridization powers between DNA molecules to perform various computing tasks in DNA computing and DNA storage. Unlike specific hybridization, non-specific hybridization provides a natural way to measure similarity between the objects represented by different DNA sequences. We utilize such property to build an instance-based learning model which recognizes an object by its similarity with other samples. The handwriting digit images in MNIST dataset are encoded by DNA sequences using a deep learning encoder. And the reverse complement sequence of a query image is used to hybridize with the training instance sequences. Simulation results by NUPACK show that this classification model by DNA could achieve 95% accuracy on average. Wet-lab experiments also validate the predicted yield is consistent with the hybridization strength. Our work proves that it is feasible to build an effective instance-based classification model for practical application.

IDENTIFYING POSSIBLE RUMOR SPREADERS ON TWITTER USING THE SVM AND FEATURE LEVEL EXTRACTION

In everyday life, many events occur and give rise to various kinds of information, which are also rumors. Rumors can cause fear and influence public opinion about the event in question. Identifying possible rumor spreaders is extremely helpful in preventing the spread of rumors. Feature extraction can be done to expand the feature set, which consists of conversational features in the form of social networks formed from user replies, user features such as following, tweet count, verified, etc., and tweet features with text analysis such as punctuation and sentiment values. These features become instances used for classification. This study aims to identify possible spreaders of rumors on Twitter with the SVM classification model. This instance-based classification algorithm is good for linear and non-linear classification. In the non-linear classification, additional kernels are used, such as linear, RBF, and sigmoid. The research focuses on getting the best model with high performance values from all the models and kernel functions that have been defined. It was found that the SVM classification model with the RBF kernel has a high overall performance value for each data combination with a ratio of the amount of data is 1:1 or the difference is very large. This model gives accurate results with an average of 97.02%. With a wide distribution of data, the SVM classification model with the RBF kernel is able to map the data properly.

Detecting block ciphers generic attacks: An instance-based machine learning method

Cryptography facilitates selective communication through encryption of messages and or data. Block-cipher processing is one of the prominent methods for modern cryptographic symmetric encryption schemes. The rise in attacks on block-ciphers led to the development of more difficult encryption schemes. However, attackers decrypt block-ciphers through generic attacks given sufficient time and computing. Recent research had applied machine learning classification algorithms to develop intrusion detection systems to detect multiple types of attacks. These intrusion detection systems are limited by misclassifying generic attacks and suffer reduced effectiveness when evaluated for detecting generic attacks only. Hence, this study introduced and proposed k-nearest neighbors, an instance-based machine learning classification algorithm, for the detection of generic attacks on block-ciphers. The value of k was varied (i.e., 1, 3, 5, 7, and 9) and multiple nearest neighbors classification models were developed and evaluated using two distance functions (i.e., Manhattan and Euclidean) for classifying between generic attacks and normal network packets. All nearest neighbors models using the Manhattan distance function performed better than their Euclidean counterparts. The 1-nearest neighbor (Manhattan distance function) model had the highest overall accuracy of 99.6%, a generic attack detection rate of 99.5% which tallies with the 5, 7, and 9 nearest neighbors models, and a false alarm rate of 0.0003 which is the same for all Manhattan nearest neighbors classification models. These instance-based methods performed better than some existing methods that even implemented an ensemble of deep-learning algorithms. Therefore, an instance-based method is recommended for detecting block-ciphers generic attacks.

HITS: Binarizing physiological time series with deep hashing neural network

• A deep neural network architecture is proposed to learn binary hash codes from physiological time series. • High utility, similarity preserving and temporal relationship are jointly learned with multiple designed objectives. • Experiments on ECG and EEG datasets show the effectiveness of the proposed method. In this paper, we aim to transform numerical physiological time series into binary hash codes, that can be further used for indexing large-scale dataset, or accelerating downstream tasks such as instance-based classification . We propose HITS to learn binary H ash codes from phys I ological T ime S eries. HITS first builds a very deep one-dimensional convolutional neural network to learn lower-dimensional representations from raw physiological time series. Then, HITS jointly learns high utility, similarity preserving, and temporal related compact binary codes by corresponding objectives with imposed. Finally, given a new physiological time series, HITS can encode it to a binary hash code. Experiments are performed on two real-world Electrocardiogram and Electroencephalogram datasets. The accuracy of a k-nearest classifier is used to evaluate the quality of codes. HITS outperforms the second-best baseline 7.42% on average of k = { 1 , 2 , 4 , 8 , 16 , 32 } with code length c = 48 , while reducing 45.51% searching time than the same length numerical vectors on average of c = { 16 , 24 , 32 , 48 , 64 } . HITS consistently achieve higher classification accuracy than compared methods using k-nearest classifier varying different k and code length. Thus, HITS learns better binary hash codes than compared methods.

Fast data reduction by space partitioning via convex hull and MBR computation

Large volumes of training data introduce high computational cost in instance-based classification. Data reduction algorithms select or generate a small (condensing) set of representative training prototypes from the available training data. The Reduction by Space Partitioning algorithm is one of the most well-known prototype generation algorithms that repetitively divides the original training data into subsets. This partitioning process needs to identify the diameter of each subset, i.e., its two farthest instances. This is a costly process since it requires the calculation of all distances between the instances in each subset. The paper introduces two new very fast variations that, instead of computing the actual diameter of a subset, choose a pair of distant-enough instances. The first variation uses instances belonging to an exact 3d convex hull of the subset, while the second one uses instances belonging to the minimum bounding rectangle of the subset. Our experimental study shows that the new variations vastly outperform the original algorithm without a penalty in classification accuracy and reduction rate.

Using Proximity Graph Cut for Fast and Robust Instance-Based Classification in Large Datasets

K-nearest neighbours (kNN) is a very popular instance-based classifier due to its simplicity and good empirical performance. However, large-scale datasets are a big problem for building fast and compact neighbourhood-based classifiers. This work presents the design and implementation of a classification algorithm with index data structures, which would allow us to build fast and scalable solutions for large multidimensional datasets. We propose a novel approach that uses navigable small-world (NSW) proximity graph representation of large-scale datasets. Our approach shows 2–4 times classification speedup for both average and 99th percentile time with asymptotically close classification accuracy compared to the 1-NN method. We observe two orders of magnitude better classification time in cases when method uses swap memory. We show that NSW graph used in our method outperforms other proximity graphs in classification accuracy. Our results suggest that the algorithm can be used in large-scale applications for fast and robust classification, especially when the search index is already constructed for the data.

Cooperative coevolutionary instance selection for multilabel problems

Multilabel classification as a data mining task has recently attracted greater research interest. Many current data mining applications address problems having instances that belong to more than one class, which requires the development of new efficient methods. Advantageously using the correlation among different labels can provide better performance than methods that deal with each label separately. Instance-based classification models, such as k-nearest neighbors for multilabel datasets, ML-kNN, are among the best performing methods on any classification task and have also shown very goog performance in multilabel problems. Despite their simplicity, they achieve comparable performance to considerably more complex methods. One of the challenges associated with instance-based classification models is their requirement for storing all the training instances in memory. To ameliorate this problem, instance selection methods have been proposed. However, their application to multilabel problems is problematic because the adaptation of most of their concepts to multilabel problems is difficult. In this paper, we propose a cooperative coevolutionary algorithm for instance selection for multilabel problems. Two different populations evolve together cooperatively. One of the populations is devoted to obtaining solutions for each label, whereas the other population combines these results into solutions for the instance selection for multilabel dataset tasks. On a large set of 70 real-world problems, our approach improves the results of both the ML-kNN method with the whole dataset and an instance selection method using a standard evolutionary algorithm.

Instance-Based Classification Through Hypothesis Testing

Classification is a fundamental problem in machine learning and data mining. During the past decades, numerous classification methods have been presented based on different principles. However, most existing classifiers cast the classification problem as an optimization problem and do not address the issue of statistical significance. In this paper, we formulate the binary classification problem as a two-sample testing problem. More precisely, our classification model is a generic framework that is composed of two steps. In the first step, the distance between the test instance and each training instance is calculated to derive two distance sets. In the second step, the two-sample test is performed under the null hypothesis that the two sets of distances are drawn from the same cumulative distribution. After these two steps, we have two ${p}$ -values for each test instance and the test instance is assigned to the class associated with the smaller ${p}$ -value. Essentially, the presented classification method can be regarded as an instance-based classifier based on hypothesis testing. The experimental results on 38 real data sets show that our method is able to achieve the same level performance as several classic classifiers and has significantly better performance than existing testing-based classifiers. Furthermore, we can handle outlying instances and control the false discovery rate of test instances assigned to each class under the same framework.

On the generation of multi-label prototypes

Data reduction techniques play a key role in instance-based classification to lower the amount of data to be processed. Prototype generation aims to obtain a reduced training set in order to obtain accurate results with less effort. This translates into a significant reduction in both algorithms’ spatial and temporal burden. This issue is particularly relevant in multi-label classification, which is a generalization of multiclass classification that allows objects to belong to several classes simultaneously. Although this field is quite active in terms of learning algorithms, there is a lack of data reduction methods. In this paper, we propose several prototype generation methods from multi-label datasets based on Granular Computing. The simulations show that these methods significantly reduce the number of examples to a set of prototypes without significantly affecting classifiers’ performance.

Bagging-based instance selection for instance-based classification

Bagging-based instance selection for instance-based classification

Instance-based classification using prototypes generated from large noisy and streaming datasets

Nowadays, large volumes of training data are available from various data sources and streaming environments. Instance-based classifiers perform adequately when they use only a small subset of such datasets. Larger data volumes introduce high computational cost that prohibits the timely execution of the classification process. Conventional prototype selection and generation algorithms are also inappropriate for data streams and large datasets. In the past, we proposed prototype generation algorithms that maintain a dynamic set of prototypes and are appropriate for such types of data. Dynamic because existing prototypes may be updated, or new prototypes may be appended to the set of prototypes in the course of processing. Still, repetitive generation of new prototypes may result to forming unpredictably large sets of prototypes. In this paper, we propose a new variation of our algorithm that maintains the prototypes in a convenient and manageable way. This is achieved by removing the weakest prototype when a new prototype is generated. The new algorithm has been tested on several datasets. The experimental results reveal that it is as accurate as its predecessor, yet it is more efficient and noise tolerant.

PERBANDINGAN KINERJA MUTUAL K-NEAREST NEIGHBOR (MKNN) DAN K-NEAREST NEIGHBOR (KNN) DALAM ANALISIS KLASIFIKASI KELAYAKAN KREDIT

Credit feasibility analysis is important for lenders to avoid the risk among the increasement of credit applications. This analysis can be carried out by the classification technique. Classification technique used in this research is instance-based classification. These techniques tend to be simple, but are very dependent on the determination of K values. K is number of nearest neighbor considered for class classification of new data. A small value of K is very sensitive to outliers. This weakness can be overcome using an algorithm that is able to handle outliers, one of them is Mutual K-Nearest Neighbor (MKNN). MKNN removes outliers first, then predicts new observation classes based on the majority class of their mutual nearest neighbors. The algorithm will be compared with KNN without outliers. The model is evaluated by 10-fold cross validation and the classification performance is measured by Gemoetric-Mean of sensitivity and specificity. Based on the analysis the optimal value of K is 9 for MKNN and 3 for KNN, with the highest G-Mean produced by KNN is equal to 0.718, meanwhile G-Mean produced by MKNN is 0.702. The best alternative to classifying credit feasibility in this study is K-Nearest Neighbor (KNN) algorithm with K=3.Keywords: Classification, Credit, MKNN, KNN, G-Mean.

Arabic Authorship Attribution

Law enforcement faces problems in tracing the true identity of offenders in cybercrime investigations. Most offenders mask their true identity, impersonate people of high authority, or use identity deception and obfuscation tactics to avoid detection and traceability. To address the problem of anonymity, authorship analysis is used to identify individuals by their writing styles without knowing their actual identities. Most authorship studies are dedicated to English due to its widespread use over the Internet, but recent cyber-attacks such as the distribution of Stuxnet indicate that Internet crimes are not limited to a certain community, language, culture, ideology, or ethnicity. To effectively investigate cybercrime and to address the problem of anonymity in online communication, there is a pressing need to study authorship analysis of languages such as Arabic, Chinese, Turkish, and so on. Arabic, the focus of this study, is the fourth most widely used language on the Internet. This study investigates authorship of Arabic discourse/text, especially tiny text, Twitter posts. We benchmark the performance of a profile-based approach that uses n -grams as features and compare it with state-of-the-art instance-based classification techniques. Then we adapt an event-visualization tool that is developed for English to accommodate both Arabic and English languages and visualize the result of the attribution evidence. In addition, we investigate the relative effect of the training set, the length of tweets, and the number of authors on authorship classification accuracy. Finally, we show that diacritics have an insignificant effect on the attribution process and part-of-speech tags are less effective than character-level and word-level n -grams.

A weighted pattern matching approach for classification of imbalanced data with a fireworks-based algorithm for feature selection

ABSTRACTLearning a classifier from imbalanced data is a challenging problem in Machine learning. A dataset is said to be imbalanced when the number of instances belonging to one class is much less than the number of instances belonging to the other class. Classifiers that proves efficient on standard data fail when the data is imbalanced as they are over trained by the majority class instances. Since class imbalance is a common characteristic of real-world data, the need for better classifiers becomes essential. This paper proposes a novel instance-based classification algorithm called Weighted Pattern Matching based Classification (PMC+) for classifying imbalanced data. PMC+ classifies unlabelled instances by computing the absolute difference between the feature values of the instances in the dataset and the unlabelled instance. PMC+ employs a simple classification procedure with weights and shows reasonably good performance. To improve the performance of PMC+, Fireworks based Feature and Weight Selection algorithm based on the idea of PMC+ has been proposed. PMC+ is evaluated on 44 binary imbalanced datasets and 15 multiclass imbalanced datasets. Although PMC+ does not employ a resampling or cost-sensitive method, experiments show that PMC+ is effective for classification of imbalanced data. The results of the experiments were validated using various non-parametric statistical tests.

Instance-based classification with Ant Colony Optimization

Instance-based learning (IBL) methods predict the class label of a new instance based directly on the distance between the new unlabeled instance and each labeled instance in the training set, without constructing a classification model in the training phase. In this paper, we introduce a novel class-based feature weighting technique, in the context of instance-based distance methods, using the Ant Colony Optimization meta-heuristic. We address three different approaches of instance-based classification: k-Nearest Neighbours, distance-based Nearest Neighbours, and Gaussian Kernel Estimator. We present a multi-archive adaptation of the ACO? algorithm and apply it to the optimization of the key parameter in each IBL algorithm and of the class-based feature weights. We also propose an ensemble of classifiers approach that makes use of the archived populations of the ACO? algorithm. We empirically evaluate the performance of our proposed algorithms on 36 benchmark datasets, and compare them with conventional instance-based classification algorithms, using various parameter settings, as well as with a state-of-the-art coevolutionary algorithm for instance selection and feature weighting for Nearest Neighbours classifiers.

Differentially private nearest neighbor classification

Instance-based learning, and the k-nearest neighbors algorithm (k-NN) in particular, provide simple yet effective classification algorithms for data mining. Classifiers are often executed on sensitive information such as medical or personal data. Differential privacy has recently emerged as the accepted standard for privacy protection in sensitive data. However, straightforward applications of differential privacy to k-NN classification yield rather inaccurate results. Motivated by this, we develop algorithms to increase the accuracy of private instance-based classification. We first describe the radius neighbors classifier (r-N) and show that its accuracy under differential privacy can be greatly improved by a non-trivial sensitivity analysis. Then, for k-NN classification, we build algorithms that convert k-NN classifiers to r-N classifiers. We experimentally evaluate the accuracy of both classifiers using various datasets. Experiments show that our proposed classifiers significantly outperform baseline private classifiers (i.e., straightforward applications of differential privacy) and executing the classifiers on a dataset published using differential privacy. In addition, the accuracy of our proposed k-NN classifiers are at least comparable to, and in many cases better than, the other differentially private machine learning techniques.

An efficient algoritm for classification of EEG eye state data

AbstractA recent technology which makes possible for us to interact with automated systems without using any body part is called Brain Computer Interface (BCI). In its concrete applications, electroencephalogram (EEG) is benefited by a BCI environment for being capable of obtaining brain waves. In our study, evaluation of success rates of the predictions made by C x k - Nearest Neighborhood (Cxk-NN) Algorithm for EEG Eye State Data whose states are called “Opened Eye“ and “Closed Eye“ is applied. This EEG Eye State dataset is obtained from UCI Machine Learning Repository on the web and it is a highly-used benchmark data on this field. As there are only two classes of the signals, we test binary classification performance of our classification algorithm (Cxk –NN). Comparison of those values with the ones obtained by the other successful classification algorithms in the literature applied on the same data set also take place in our study. Cxk-NN is an instance-based classification method advanced from simple k – Nearest Neighborhood Algorithm, and improved success results are observed when it is compared with k-NN.
 Keywords: brain computer interface (bci), classificaiton, eeg, c x k - nearest neighborhood algorihtm.

Volume estimation of Cryptomeria japonica logs in southern Brazil using artificial intelligence models

This study aimed to test taper functions and artificial intelligence (AI) models in order to estimate merchantable volumes of Japanese cedar (Cryptomeria japonica) trees in a homogenous plantation in southern Brazil. A total of 30 individuals were rigorously scaled and their total volumes were calculated, including those of the following log assortments: veneer, sawn, pulp and energy. Three AI models, i.e. two variants of k-nearest neighbours (KNN) instance-based classification (one and three nearest neighbours) and an artificial neural network (ANN) approach, were compared with three traditional taper models: fifth-order polynomial, fractional powers and the Garay model. The estimated volumes were compared with the actual volumes by means of the standard error (Syx), bias, precision and accuracy. Total volume estimates proved to be unbiased (maximum bias 5.42%), precise (maximum precision 9.28%) and accurate (maximum accuracy 10.79%) with all of the investigated models. The tested models tended to give lower bias, better precision and accuracy in the middle portion of the stems, but worse estimates at the base and tip (maximum bias −12.41%). In general, the KNN models improved merchantable volume estimation, particularly KNN1, which is a straightforward and simple method. We conclude that AI techniques have appeal for application in forest inventories and that KNN is a particularly interesting alternative for tree volume estimation.

Judgment Analysis in a Dynamic Multitask Environment: Capturing Nonlinear Policies Using Decision Trees

Policy capturing is a judgment analysis method that typically uses linear statistical modeling to estimate expert judgments. A variant to this technique is to capture decision policies using data-mining algorithms designed to handle nonlinear decision rules, missing attributes, and noisy data. In the current study, we tested the effectiveness of a decision-tree induction algorithm and an instance-based classification method for policy capturing in comparison to the standard linear approach. Decision trees are relevant in naturalistic decision-making contexts since they can be used to represent “fast-and-frugal” judgment heuristics, which are well suited to describe human cognition under time pressure. We examined human classification behavior using a simulated naval air defense task in order to empirically compare the C4.5 decision-tree algorithm, the k-nearest neighbors algorithm, and linear regression on their ability to capture individual decision policies. Results show that C4.5 outperformed the other methods in terms of goodness of fit and cross-validation accuracy. Decision-tree models of individuals’ judgment policies actually classified contacts more accurately than their human counterparts, resulting in a threefold reduction in error rates. We conclude that a decision-tree induction algorithm can yield useful models for training and decision support applications, and we discuss the application of judgmental bootstrapping in real time in dynamic environments.