Instance Selection Algorithms Research Articles

A distributed evolutionary based instance selection algorithm for big data using Apache Spark

Instance selection is an important preprocessing technology in data mining and machine learning. In this paper, we proposed a novel evolutionary based instance selection algorithm for big data. First, we defined a coarse granularity chromosome structure to reduce the size of search space and costs of chromosome operations(recombination and mutation, etc.). Then a stratified evolution strategy was proposed to remove the hyper parameter in classic fitness function and achieve precise control over the reduction ratio of instances. Finally, a sampling-based fitness function was proposed to reduce the time complexity. Experimental results shown that our new algorithm is efficient to complete the instance selection task on data set with millions of instances in minutes-level. The 10-fold cross-validation also proved that the selection results on many datasets have high nearest neighbor classification accuracy.

Active learning strategies for robotic tactile texture recognition tasks.

Accurate texture classification empowers robots to improve their perception and comprehension of the environment, enabling informed decision-making and appropriate responses to diverse materials and surfaces. Still, there are challenges for texture classification regarding the vast amount of time series data generated from robots' sensors. For instance, robots are anticipated to leverage human feedback during interactions with the environment, particularly in cases of misclassification or uncertainty. With the diversity of objects and textures in daily activities, Active Learning (AL) can be employed to minimize the number of samples the robot needs to request from humans, streamlining the learning process. In the present work, we use AL to select the most informative samples for annotation, thus reducing the human labeling effort required to achieve high performance for classifying textures. We also use a sliding window strategy for extracting features from the sensor's time series used in our experiments. Our multi-class dataset (e.g., 12 textures) challenges traditional AL strategies since standard techniques cannot control the number of instances per class selected to be labeled. Therefore, we propose a novel class-balancing instance selection algorithm that we integrate with standard AL strategies. Moreover, we evaluate the effect of sliding windows of two-time intervals (3 and 6 s) on our AL Strategies. Finally, we analyze in our experiments the performance of AL strategies, with and without the balancing algorithm, regarding f1-score, and positive effects are observed in terms of performance when using our proposed data pipeline. Our results show that the training data can be reduced to 70% using an AL strategy regardless of the machine learning model and reach, and in many cases, surpass a baseline performance. Finally, exploring the textures with a 6-s window achieves the best performance, and using either Extra Trees produces an average f1-score of 90.21% in the texture classification data set.

Manifold embedded instance selection to suppress negative transfer in motor imagery-based brain–computer interface

Individual distribution discrepancy poses significant challenges to cross-subject electroencephalography (EEG) signal decoding. Although transfer learning has emerged as an effective approach to minimize this distribution gap, EEG signals’ low voltage amplitude makes them vulnerable to noise and results in abnormal, low-quality samples that cause negative transfer phenomenon. This phenomenon, in turn, undermines the efficacy of transfer learning and impedes brain-computer interface (BCI) applications. To overcome this challenge, we introduce the manifold embedded instance selection (MEIS) algorithm, which addresses negative transfer. The MEIS algorithm operates in two ways: converting raw EEG matrices into manifold embedded vectors that maintain sample discriminability, and designing an evaluator to assess the transferability of samples and filter out negative transfer samples from the source domain. When faced with a large number of source domains, our proposed method employs domain similarity estimation to determine the most beneficial subset of source domains for the target domain. The effectiveness of the proposed method is confirmed via offline and simulated online motor imagery-based BCI experiments where it demonstrates superior accuracy compared to other advanced techniques. Additionally, the MEIS algorithm significantly reduces both training time and the number of required training samples without affecting the model’s performance. The code is available at https://github.com/ZilinL/MEIS.

Instance and Feature Selection Using Fuzzy Rough Sets: A Bi-Selection Approach for Data Reduction

Data reduction, aiming to reduce the original data by selecting the most representative information, is an important technique of preprocessing data. At present, large-scale or huge data are very common and the development of data reduction techniques for such data has attracted much attention. As a powerful tool for handling uncertainty in real-valued data, the fuzzy rough set theory has been widely applied to data reduction including extensive feature selection methods and some instance selection approaches. Nevertheless, not much work has been devoted to the simultaneous selection of feature and instance based on fuzzy rough sets. In this paper, we investigate the fuzzy rough set-based bi-selection issue for data reduction. Specifically, the unified concepts of the importance degrees of fuzzy granules are presented to select the representative instances first and then the critical features. An instance selection algorithm with a noise elimination technique is provided to firstly remove the noise and then select the representative instances according to the importance degrees of fuzzy granules. Then, the importance-degree-preserved attribute reduction is proposed, and a corresponding feature selection algorithm with a wrapper technique is given to search for a best feature subset. Lastly, the bi-selection method based on fuzzy rough sets (BSFRS) is presented for data reduction by integrating the instance selection and the feature selection methods. Moreover, some numerical experiments are conducted to assess the performance of BSFRS, and the results show that BSFRS performs well in terms of the effectiveness.

A stochastic approximation approach to fixed instance selection

Instance selection plays a critical role in enhancing the efficacy and efficiency of machine learning tools when utilised for a data mining task. This study proposes a fixed instance selection algorithm based on simultaneous perturbation stochastic approximation that works in conjunction with any supervised machine learning method and any corresponding performance metric, which we call SpFixedIS. This algorithm provides an approximate solution to the NP-hard instance selection problem and additionally serves as a way of intelligently selecting a specified number of instances within a training set with regards to a machine learning model. The shape of the objective function obtained from the test accuracy against the number of instances selected is examined extensively for our instance selection algorithm. The SpFixedIS algorithm was tested on 43 diverse datasets across 6 different machine learning classifiers. The results show that in over 90% of cases SpFixedIS provides a statistically significant improvement at a 5% level with intelligent selection over random selection for the same number of instances. Furthermore, with respect to probabilistic models, specifically Gaussian Naive Bayes, SpFixedIS provides a statistically significant improvement compared to models that utilise the entirety of the training set in 84% of the experimented values ranging from 50 to 1000 instances.

Instance selection using one‐versus‐all and one‐versus‐one decomposition approaches in multiclass classification datasets

AbstractInstance is important in data analysis and mining; it filters out unrepresentative, redundant, or noisy data from a given training set to obtain effective model learning. Various instance selection algorithms are proposed in the literature, and their potential and applicability in data cleaning and preprocessing steps are demonstrated. For multiclass classification datasets, the existing instance selection algorithms must deal with all the instances across the different classes simultaneously to produce a reduced training set. Generally, every multiclass classification dataset can be regarded as a complex domain problem, which can be effectively solved using the divide‐and‐conquer principle. In this study, the one‐versus‐all (OVA) and one‐versus‐one (OVO) decomposition approaches were used to decompose a multiclass dataset into multiple binary class datasets. These approaches have been widely employed when constructing the classifier but have never been considered in instance selection. The results of instance selection performance obtained with the OVA, OVO, and baseline approaches were assessed and compared for 20 different domain multiclass datasets as the first study and five medical domain datasets as the validation study. Furthermore, three instance selection algorithms were compared, including IB3, DROP3, and GA. The results demonstrate that using the OVO approach to perform instance selection can make the support vector machine (SVM) and k‐nearest neighbour (k‐NN) classifiers perform significantly better than the OVA and baseline approaches in terms of the area under the ROC curve (AUC) rate, regardless of the instance selection algorithm used. Moreover, the OVO approach can provide reasonably good data reduction rates and processing times, which are all better than those of the OVA approach.

Robust bag classification approach for multi-instance learning via subspace fuzzy clustering

Multi-instance learning (MIL) allows predictive algorithms to use complex data representation. The data in MIL is organized in the form of labeled bags of instances, and the labels of instances are not available in the training phase. The processing and classification of complex bag representation result in a complicated hypothesis space. Moreover, identifying essential instances inside is also important, as these instances trigger positive labels for the bag and play a vital role in the model interpretation and bag classification. The recent MIL algorithms are not robust to tackle hypothesis space complication. Additionally, the existing instance selection algorithms are based on explicit assumptions regarding the relationship of instances to the bag label. However, these assumptions may hold for a specific bag in the dataset but may not apply to the whole dataset. To deal with the hypothesis space complication and robust instance selection without any prior assumption, this paper proposes a fuzzy subspace clustering approach for robust instance selection and ensemble-based variant for Fisher vector (FV) encoding to solve MIL problems, named (FCBE-miFV). Specifically, the proposed algorithm uses a subspace fuzzy clustering approach to compute instance selection probabilities, selects essential instances from the bag, transforms the input bag into FV, and classifies the generated bag encodings using a stacking-based ensemble approach to obtain improved bag level classification performance. The proposed FCBE-miFV improves model performance by incorporating essential instances in the bag encoding process. The experimental results show that the FCBE-miFV obtained comparable performance to the state-of-the-art MIL problems.

A New Instance Selection Method for Enlarging Margins Between Classes

As discarding superfluous instances in data sets shortens the learning process, it also increases learning performance because of eliminating noisy data. Instance selection methods are commonly utilized to undertake the abovementioned tasks. In this paper, we propose a new supervised instance selection algorithm called Border Instances Reduction using Classes Handily (BIRCH). BIRCH considers k-nearest neighbors of each instance and selects instances that have neighbors from the only same class, namely, but not having neighbors from the different classes. It has been compared with one traditional and four state-of-the-art instance selection algorithms by using fifteen data sets from various domains. The empirical results show BIRCH well delivers the trade-off between accuracy rate and reduction rate by tuning the number of neighbors. Furthermore, the proposed method guarantees to yield a high classification accuracy. The source code of the proposed algorithm can be found in https://github.com/fatihaydin1/BIRCH.

Evolutionary Instance Selection Based on Preservation of the Data Probability Density Function

The generation of massive amounts of data has motivated the use of machine learning models to perform predictive analysis. However, the computational complexity of these algorithms depends mainly on the number of training samples. Thus, training predictive models with high generalization performance within a reasonable computing time is a challenging problem. Instance selection (IS) can be applied to remove unnecessary points based on a specific criterion to reduce the training time of predictive models. This paper introduces an evolutionary IS algorithm that employs a novel fitness function to maximize the similarity of the probability density function (PDF) between the original dataset and the selected subset, and to minimize the number of samples chosen. This method is compared against six other IS algorithms using four performance measures relating to the accuracy, reduction rate, PDF preservation, and efficiency (which combines the first three indices using a geometric mean). Experiments with 40 datasets show that the proposed approach outperforms its counterparts. The selected instances are also used to train seven classifiers, in order to evaluate the generalization and reusability of this approach. Finally, the accuracy results show that the proposed approach is competitive with other methods and that the selected instances have adequate capabilities for reuse in different classifiers.

Cluster-oriented instance selection for classification problems

More training instances could lead to better classification accuracy. However, accuracy could also degrade if more training instances mean further noises and outliers. Additional training instances arguably need additional computational resources in future data mining operations. Instance selection algorithms identify subsets of training instances that could desirably increase accuracy or at least do not decrease accuracy significantly. There exist many instance selection algorithms, but no single algorithm, in general, dominates the others. Moreover, existing instance selection algorithms do not allow direct controlling of the instance selection rate. In this paper, we present a simple and generic cluster-oriented instance selection algorithm for classification problems. Our proposed algorithm runs an unsupervised K Means Clustering algorithm on the training instances and with a given selection rate, selects instances from the centers and the borders of the clusters. On 24 benchmark classification problems, when very similar percentages of instances are selected by various instance selection algorithms, K Nearest Neighbours classifiers achieve more than 2%–3% better accuracy when using instances selected by our proposed method than when using those selected by other state-of-the-art generic instance selection algorithms.

Instance selection for big data based on locally sensitive hashing and double-voting mechanism

The increasing data volumes impose unprecedented challenges to traditional data mining in data preprocessing, learning, and analyzing, it has attracted much attention in designing efficient compressing, indexing and searching methods recently. Inspired by locally sensitive hashing (LSH), divide-and-conquer strategy, and double-voting mechanism, we proposed an iterative instance selection algorithm, which needs to run p rounds iteratively to reduce or eliminate the unwanted bias of the optimal solution by double-voting. In each iteration, the proposed algorithm partitions the big dataset into several subsets and distributes them to different computing nodes. In each node, the instances in local data subset are transformed into Hamming space by l hash function in parallel, and each instance is assigned to one of the l hash tables by the corresponding hash code, the instances with the same hash code are put into the same bucket. And then, a proportion of instances are randomly selected from each hash bucket in each hash table, and a subset is obtained. Thus, totally l subsets are obtained, which are used for voting to select the locally optimal instance subset. The process is repeated p times to obtain p subsets. Finally, the globally optimal instance subset is obtained by voting with the p subsets. The proposed algorithm is implemented with two open source big data platforms, Hadoop and Spark, and experimentally compared with three state-of-the-art methods on testing accuracy, compression ratio, and running time. The experimental results demonstrate that the proposed algorithm provides excellent performance and outperforms three baseline methods.

A multi-objective evolutionary algorithm based on length reduction for large-scale instance selection

Instance selection, as an important data pre-processing task, is widely used in supervised classification. Recently, a series of instance selection algorithms with different techniques have been suggested. Among them, evolutionary algorithms (EAs) have shown competitive performance. However, when the size of instance set is large, these EA-based algorithms may face great challenges on search efficiency and computational cost. To this end, in this paper, a multi-objective evolutionary algorithm based on length reduction, termed as LRIS, is proposed for large-scale instance selection, where a length reduction strategy is suggested to recursively shorten the length of each individual in the population, and improve the computational efficiency of LRIS greatly. Specifically, in the proposed length reduction strategy of LRIS, each gene in the individuals has a probability of being deleted, whose probability is obtained according to the importance of the corresponding instance in the instance set and the importance of the corresponding gene in the population. Then, two evolutionary operators (e.g. crossover and mutation) based on the length reduction strategy are developed to generate offspring population from the reduced population. In addition, an individual repairing operator is also designed to repair the length of over-reduced individuals. Experimental results on 12 large-scale data sets have demonstrated the efficiency and the effectiveness of the proposed LRIS in comparison with the state-of-the-art EA-based instance selection algorithms.

Data cleaning issues in class imbalanced datasets: instance selection and missing values imputation for one-class classifiers

PurposeClass imbalance learning, which exists in many domain problem datasets, is an important research topic in data mining and machine learning. One-class classification techniques, which aim to identify anomalies as the minority class from the normal data as the majority class, are one representative solution for class imbalanced datasets. Since one-class classifiers are trained using only normal data to create a decision boundary for later anomaly detection, the quality of the training set, i.e. the majority class, is one key factor that affects the performance of one-class classifiers.Design/methodology/approachIn this paper, we focus on two data cleaning or preprocessing methods to address class imbalanced datasets. The first method examines whether performing instance selection to remove some noisy data from the majority class can improve the performance of one-class classifiers. The second method combines instance selection and missing value imputation, where the latter is used to handle incomplete datasets that contain missing values.FindingsThe experimental results are based on 44 class imbalanced datasets; three instance selection algorithms, including IB3, DROP3 and the GA, the CART decision tree for missing value imputation, and three one-class classifiers, which include OCSVM, IFOREST and LOF, show that if the instance selection algorithm is carefully chosen, performing this step could improve the quality of the training data, which makes one-class classifiers outperform the baselines without instance selection. Moreover, when class imbalanced datasets contain some missing values, combining missing value imputation and instance selection, regardless of which step is first performed, can maintain similar data quality as datasets without missing values.Originality/valueThe novelty of this paper is to investigate the effect of performing instance selection on the performance of one-class classifiers, which has never been done before. Moreover, this study is the first attempt to consider the scenario of missing values that exist in the training set for training one-class classifiers. In this case, performing missing value imputation and instance selection with different orders are compared.

Effect of the distance functions on the distance-based instance selection for the feed-forward neural network

The distance-based instance selection algorithm for the feed-forward neural network is a numerosity reduction technique. It selects only the instances at the decision boundary between consecutive classes of data to lessen the number of instances in the original training set using the Euclidean distance function. This paper studies improvement of the reduction performance and the classification performance of the nine distance functions in the distance-based instance selection algorithm. The evaluation was conducted on the real-world data sets from the UCI machine learning repository and ELENA project. The data reduction performance results confirmed that the Chebyshev, Cosine, and Minkowski distance functions are recommended for the integer data type. The Minkowski distance function is recommended for the categorical data type. The Jaccard distance function is recommended for the real data type. The selection of the initial distance function based on the data type can make the distance-based instance selection algorithm produce the best classification performance.

An Improved under Sampling Approaches for Concept Drift and Class Imbalance Data Streams using Improved Cuckoo Search Algorithm

One of the biggest challenges in the recent times in the field of data stream learning is to mitigate the presence of concept drift. There are numerous challenges in overcoming the concept drift, such as changing class ratio, huge volume of data and real time processing for effective knowledge discovery. Evolutionary search techniques are one of the new paradigms to handle huge dimensionality and scalability of the data streams. One of the finest and least applied evolutionary search approaches is the cuckoo search technique for data streams. To solve both the concept drift and class imbalance issues simultaneously, in this paper we have proposed an approach using nature inspired evolutionary optimizing technique known as Cuckoo Feature and Instance Selection (CFIS) algorithm. The performance evaluation of the proposed approach is done on an exclusive experimental setup of 15 data streams formed and compared with two data stream approach. Moreover, a set of six evaluation criteria’s are considered for showing overall better performance of the proposed approach in the presence of concept drift and class imbalance.

Combining feature selection, instance selection, and ensemble classification techniques for improved financial distress prediction

Bankruptcy prediction and credit scoring are major problems in financial distress prediction. Studies have shown that prediction models can be made more effective by performing data preprocessing procedures. Moreover, classifier ensembles are likely to outperform single classifiers. Although feature selection, instance selection, and classifier ensembles are known to affect final prediction results, their combined effects on bankruptcy prediction and credit scoring problems have not been fully explored. This study compares the performance of three feature selection algorithms, three instance selection algorithms, four classification algorithms, and two ensemble learning techniques. The results obtained using five bankruptcy prediction and five credit scoring datasets indicate that by carefully considering the combination of these three factors, better prediction models can be developed than by considering only one related factor.

Representativeness-Based Instance Selection for Intrusion Detection

With the continuous development of network technology, an intrusion detection system needs to face detection efficiency and storage requirement when dealing with large data. A reasonable way of alleviating this problem is instance selection, which can reduce the storage space and improve intrusion detection efficiency by selecting representative instances. An instance is representative not only in its class but also in different classes. This representativeness reflects the importance of an instance. Since the existing instance selection algorithm does not take into account the above situations, some selected instances are redundant and some important instances are removed, increasing storage space and reducing efficiency. Therefore, a new representativeness of instance is proposed and considers not only the influence of all instances of the same class on the selected instance but also the influence of instances of different classes on the selected instance. Moreover, it considers the influence of instances of different classes as an advantageous factor. Based on this representativeness, two instance selection algorithms are proposed to handle balanced and imbalanced data problems for intrusion detection. One is a representative-based instance selection for balanced data, which is named RBIS and selects the same proportion of instances from each class. The other is a representative-based instance selection for imbalanced data, which is named RBIS-IM and selects important majority instances according to the number of instances of the minority class. Compared with other algorithms on the benchmark data sets of intrusion detection, experimental results verify the effectiveness of the proposed RBIS and RBIS-IM algorithms and demonstrate that the proposed algorithms can achieve a better balance between accuracy and reduction rate or between balanced accuracy and reduction rate.

IFSB-ReliefF: A New Instance and Feature Selection Algorithm Based on ReliefF

IFSB-ReliefF: A New Instance and Feature Selection Algorithm Based on ReliefF

Instance selection in medical datasets: A divide-and-conquer framework

Instance selection is an important problem in medical data mining. It focuses on selecting representative data samples from a given training set, whereas unrepresentative (or noisy) data samples are filtered out. This reduces the size of the training set, which then requires less storage space. In addition, when the instance selection algorithm was carefully chosen, a reduction in the training set so that it contains less noisy data can usually make the classifiers perform better than the ones without considering instance selection. In the literature, many instance selection algorithms have been proposed. However, different algorithms tend to use different criteria to determine the noisy data, making it difficult to find the best algorithm for different domain datasets. In other words, some algorithms may perform better than the others for some specific domain datasets, but may perform worse than others over other domain datasets. Instead of developing a novel algorithm that performs better than most other algorithms, this paper introduces a divide-and-conquer based instance selection (DCIS) framework that aims to improve the performance of each specific instance selection algorithm per se. Two well-known algorithms, i.e., DROP3 and IB3, are used as the baseline, and various small and large scale medical datasets are used in the experiments. Our results show that when DROP3 and IB3 are used to perform instance selection based on the DCIS framework, there is an improvement in the performance of the k-NN and SVM classifiers over the ones by the DROP3 and IB3 baselines, respectively.

Firefly algorithm for instance selection

The paper focuses on the problem of instance selection. Instance selection is currently crucial to enhance the efficacy and efficiency of machine-learning tools when they are used to solve a data-mining task and when the data are large and they are seen through the prism of the big data phenomenon. Instance selection eliminates redundant instances and thus reduces the size of the training data set. The training data, with redundant cases removed, can be more useful and ensure better performance of the final classification models. The instance selection problem belongs to the NP-hard class, so it can be solved with an approximation tool. In this paper the firefly algorithm is proposed for solving the instance selection problem. This paper is one paper, where the firefly algorithm has been used to solve a discrete optimisation problem, when in more cases previously it has been used for solving continuous optimisation problems. The firefly-based instance selection algorithm is presented and its validation is carried out. The results of the computational experiment show that the algorithm is competitive with others. The results obtained are discussed and conclusions are formulated.