Sample Reduction Methods Research Articles

Toward effective SVM sample reduction based on fuzzy membership functions

Support vector machine (SVM) is known for its good generalization performance and wide application in various fields. Despite its success, the learning efficiency of SVM decreases significantly originating from the assumption that the number of training samples increases rapidly. Consequently, the traditional SVM with standard optimization methods faces challenges such as excessive memory requirements and slow training speed, especially for large-scale training sets. To address this issue, this paper draws inspiration from the fuzzy support vector machine (FSVM). Considering that each sample has varying contributions to the decision plane, we propose an effective SVM sample reduction method based on the fuzzy membership function (FMF). This method uses FMF to calculate the fuzzy membership of each training sample. Training samples with low fuzzy memberships are then deleted. Specifically, we propose SVM sample reduction algorithms based on class center distance, kernel target alignment, centered kernel alignment, slack factor, entropy, and bilateral weighted FMF, respectively. Comprehensive experiments on UCI and KEEL datasets demonstrate that our proposed algorithms outperform other comparative methods in terms of accuracy, F-measure, and hinge-loss measures.

A general elevating framework for label noise filters

In real applications, label noise has a great influence on data modeling. As one kind of label noise treatment method, noise filter has attracted extensive attention recently. The existing filters perform well in dealing with label noise completely at random (NCAR) but poorly in dealing with the label noise in the form of clusters (LLC). Besides, the existing filters may over remove the samples located at the classification boundaries, thereby affecting the generalization performance of classifiers. To fill these gaps, we propose a general elevating framework for label noise filters. The core idea of the framework is to improve the filtering performance of the existing filters by sample reduction. Specifically, since most of the existing filters are based on classifier prediction, and the complexity of samples at the boundary will affect the prediction performance of classifiers. Therefore, reducing the complexity of the boundary sample is very helpful to improve the performance of the filters. To this end, we propose a sample reduction method, which can not only reduce the complexity of the sample at the boundary but also convert LLC to NCAR, to get some representative samples. Next, the filter based on classifier prediction is employed to recognize the noisy representative samples. Finally, the noisy labeled samples in the given data set are found according to the identified noisy representatives. Furthermore, through empirical analysis, we found that compared with some classical metrics for evaluating the performance of noise filters, classification accuracy is more suitable to measure the performance of filters. Exhaustive experiments testify the validity of the framework, and the experimental results demonstrate that the performance of our framework is especially outstanding for LLC treatment.

A new sample reduction method for decreasing the running time of the k-nearest neighbors algorithm to diagnose patients with congestive heart failure: backward iterative elimination

A new sample reduction method for decreasing the running time of the k-nearest neighbors algorithm to diagnose patients with congestive heart failure: backward iterative elimination

Development and Verification of Sample Reduction Method for Rubberized Asphalt Mixtures

The research evaluated a newly developed sample reduction method—the direct reduction procedure—aiming to minimize the variability of the reduced rubberized hot mix asphalt-gap graded (RHMA-G) mixture sample. The impact of the newly developed method was analyzed against the mechanical splitting method and the quartering method recommended in American Association of State Highway and Transportation Officials (AASHTO) R 47, Standard Practice for Reducing Samples of Asphalt Mixtures to Testing Size. The variability of three measured properties were compared between the three methods: maximum theoretical specific gravity, asphalt binder content, and aggregate gradation. The standard method of hot mix asphalt (HMA) sample reduction is not applicable to RHMA-G because RHMA-G mixes are fundamentally different than HMA. RHMA-G is gap graded and has high binder content, and RHMA-G binder is highly tacky. In addition, RHMA-G materials adhere to sample reduction tools, leading to the variability of material properties. To support this, an interlaboratory study with nine participating laboratories was conducted using a single source ¾ in. RHMA-G mixture. Participating laboratories were provided with random loose RHMA-G samples and requested to reduce samples through the three methods, and to test four replicates for each of the three mixture properties on the samples. The data was used to develop the repeatability and reproducibility limits for each of the three measured properties. As a result, data analyses indicated that the newly developed sample reduction method introduced the least variability on the three measured properties. The study led to the development of California Test 306, as the standard for sample reduction method of RHMA-G mixtures.

K-means tree: an optimal clustering tree for unsupervised learning

Tree construction is one of the popular methods for tackling any supervised task in machine learning. However, there has been little effort in applying trees for unsupervised tasks. The traditional unsupervised trees are based on recursively partitioning the space such that the achieved partitions contain similar samples. Sense of similarity depends on the models and applications. This paper tackles the issue of learning optimal clustering oblique trees for the first time and proposes a linear time algorithm for training it. Optimizing performance of infrastructures and energy consumption in the field of Internet of things can be mentioned as applications of tree and clustering, respectively. The motivation of unsupervised tree models is to preserve the data manifold, while keeping the query, while keeping the query time fast. Popular unsupervised models consist of k-d trees, random projection (RP trees), principal component analysis trees (PCA trees) and clustering trees. However, all existing methods for unsupervised tree are sub-optimal. Additionally, existing clustering trees are limited to axis-aligned trees. Further, some of the mentioned methods suffer from curse of dimensionality such as k-d trees. Despite the mentioned challenges, trees are fast in query time. On the other hand, a non-hierarchical clustering such as k-means has both: It performs well in high-dimensional problems and is locally optimal. Its learning algorithm is efficient. However, k-means clustering is not fast in query time. To address the mentioned issues, this paper proposes a novel k-means tree, a tree that outputs the centroids of clusters. The advantages of such tree are being fast in query time and also learning as good cluster centroids as k-means. As a result, problem of learning such trees is to learn both centroids and the tree parameters optimally and jointly. In this paper, this problem is first cast as a constrained minimization problem and then solved using quadratic penalty method. The method consists of learning clusters from k-means and gradually adapting centroids to the outputs of an optimal oblique tree. The alternating optimization is used, and alternation steps consist of weighted k-means clustering and tree optimization. Additionally, the training complexity of proposed algorithm is efficient. Proposed algorithm is optimal in the sense of learned clusters and tree jointly. Trees used in the k-means tree are oblique, and as per our knowledge, this is the first time that oblique trees are applied to the task of clustering. As a side product of the proposed method, sample reduction is explored and shown its merits. It is shown that computational complexity of training KMT (K-means tree) as a sample reduction method is faster than training K-means as a sample reduction. The training complexity of KMT sample reduction algorithm is logarithmic over the size of reduced train set, while training complexity of K-means is linear over the size of reduced dataset. Finally, proposed method is compared to other tree-based clustering algorithms and its superiority in terms of reconstruction error is shown. Additionally, its query complexity is compared with k-means.

A multiseed-based SVM classification technique for training sample reduction

A support vector machine (SVM) is not a popular method for a very large dataset classification because the training and testing time for such data are computationally expensive. Many researchers try to reduce the training time of SVMs by applying sample reduction methods. Many methods reduced the training samples by using a clustering technique. To reduce its high computational complexity, several data reduction methods were proposed in previous studies. However, such methods are not effective to extract informative patterns. This paper demonstrates a new supervised classification method, multiseed-based SVM (MSB-SVM), which is particularly intended to deal with very large datasets for multiclass classification. The main contributions of the paper are (i) an efficient multiseed technique for selection of seed points from circular/elongated class training samples, (ii) adjacent class pair selection from the set of multiseeds by using the minimum spanning tree, and (iii) extraction of support vectors from class pair seed equivalent regions to manage multiclass classification problems without being computationally expensive. Experimental results on a variety of datasets showed better performance compared to other sample-reducing methods in terms of training and testing time. Traditional support vector machine (SVM) solution suffers from $O(n^{2})$ time complexity, which makes it impractical for very large datasets. Here, multiseed point technique depends on the estimated density of each data, and the order of computation is $O(n$ log $n)$. Using the estimated density, the computational cost of the seed selection algorithm is $O(n)$. So, this is the only burden for reducing the sample. However, reducing the sample takes less time with the proposed algorithm compared to the clustering methods. At the same time, the number of support vectors has been abruptly reduced, which takes less time to find the decision surface. Apart from this, the classification accuracy of the proposed technique is significantly better than other existing sample reduction methods especially for large datasets.

Localization algorithm for large-scale wireless sensor networks based on FCMTSR-support vector machine

Sensor node localization is one of research hotspots in the applications of wireless sensor network field. A localization algorithm is proposed in this article which is based on improved support vector machine for large-scale wireless sensor networks. For a large-scale wireless sensor network, localization algorithm based on support vector machine faces to the problem of the large-scale learning samples. The large-scale training samples will lead to high burden of the training calculation, over learning, and low classification accuracy. In order to solve these problems, this article proposed a novel scale of training sample reduction method (FCMTSR). FCMTSR takes the training sample as point set, get the potential support vectors, and remove the non-boundary outlier data immixed by analyzing relationships between points and set. To reduce the calculation load, fuzzy C-means clustering algorithm is applied in the FCMTSR. By the FCMTSR, the training time is reduced and the localization accuracy is improved. Through the simulations, the performance of localization based on FCMTSR-support vector machine is evaluated. The results prove that the localization precision is improved 2%, the training time is reduce 55% than existing localization algorithm based on support vector machine without FCMTSR. FCMTSR-support vector machine localization algorithm also addresses the border problem and coverage hole problem effectively. Finally, the limitation of the proposed localization algorithm is discussed and future work is present.

Heuristic sample reduction method for support vector data description

Support vector data description (SVDD) has become one of the most promising methods for one-class classication fornding the boundary of the training set. However, SVDD has a time complexity of O ( N 3 ) and a space complexity of O ( N 2 ) . When dealing with very large sizes of training sets, e.g., a training set of the aeroengine gas path parameters with the size of N > 10 6 sampled from several months of ight data, SVDD fails. To solve this problem, a method called heuristic sample reduction (HSR) is proposed for obtaining a reduced training set that is manageable for SVDD. HSR maintains the classication accuracy of SVDD by building the reduced training set heuristically with the samples selected from the original. For demonstration, several articial datasets and real-world datasets are used in the experiments. In addition, a practical example of the training set of the aeroengine gas path parameters is also used to compare the performance of SVDD based on the proposed HSR with conventional SVDD and other improved methods. The experimental results are very encouraging.

A geometric density-based sample reduction method

Analysis of network traffic, financial transactions, and mobile communications are examples of applications where examining entire samples of a large dataset is computationally expensive, and requires significant memory space. A common approach to address this challenge is to reduce the number of samples without compromising the accuracy of analyzing them. In this paper, we propose a new cluster-based sample reduction method which is unsupervised, geometric, and density-based. The original data is initially divided into clusters, and each cluster is divided into “portions” defined as the areas between two concentric circles. Then, using the proposed geometric-based formulas, the selection value of each sample belonging to a specific portion is calculated. Samples are then selected from the original data according to the corresponding calculated selection value. The performance of the proposed method is measured on various datasets and compared with several clusterbased and density-based methods. We conduct various experiments on the NSL-KDD, KDDCup99, and IUSTsip datasets, and evaluate the performance of the proposed method by measuring the cluster validity indices, as well as the accuracy of the classifier applied on the reduced data. We demonstrate that the reduced dataset has similar sample scattering as that of the original dataset. We also demonstrate that, while reducing the sample size of the input dataset in half, the classification accuracy is not reduced significantly, indicating that the proposed method selects the most relevant samples from the original dataset.

Boundary detection and sample reduction for one-class Support Vector Machines

There are a large number of target data and fewer outlier data in the problem of one-class classification. The sample reduction methods used for two-class or multi-class classification cannot be applied to this problem. This paper presents a novel method to reduce the scale of training set for one-class classification. This method only preserves the sample locating near the data distribution which may become support vector. If summing the cosine value of the angle which is between the difference between the sample and the neighbor, and the difference between the sample and mean of neighbors, the cosine sum will be close to k (k is the number of the neighbors), while the sample locating near the boundary; close to 0, while the sample locating within the data distribution. Experimental results demonstrate that the proposed method can reduce the scale of training set to get faster training speed and use less support vectors in the model of one-class SVMs, and the performance does not deteriorate.

Selecting valuable training samples for SVMs via data structure analysis

In spite of its salient properties and wide acceptance, support vector machines (SVMs) still face difficulties in scalability, because solving the quadratic programming (QP) problems in SVMs training is especially costly when dealing with large sets of training data. This paper presents a new algorithm named sample reduction by data structure analysis (SR-DSA) for SVMs to improve their scalability. The SR-DSA utilizes data structure information in selecting the data points valuable in learning the separating plane. As this method is performed completely before SVMs training, it avoids the problem suffered by most sample reduction methods that choose samples heavily depending on repeated training of SVMs. Experiments on both synthetic and real world datasets show that the SR-DSA is capable of reducing the number of samples as well as the time for SVMs training while maintaining high testing accuracy.