Nonparametric Classification Method Research Articles

COMPARATIVE STUDY OF SURVIVAL SUPPORT VECTOR MACHINE AND RANDOM SURVIVAL FOREST IN SURVIVAL DATA

Survival analysis is a statistical procedure in analyzing data with the response variable is time until an event occurs (time-to-event). In the last few years, many classification approaches have been developed in machine learning, but only a few considered the presence of time-to-event variable. Random Survival Forest and Survival Support Vector Machine are machine learning approach which is a nonparametric classification method when dealing with large data and a response variable of survival time. Random Survival Forest is tree based method that using boostrapping algorithm, and Survival Support Vector Machine using hybrid approaches between regression and ranking constrain. The data used in this study is generated data in the form of right-censored survival data. This study uses the RandomForestSRC and SurvivalSVM packages on R software. This study aimed to compare the performance of the Survival Support Vector Machine and Random Survival Forest methods using simulation studies. Simulation results on right-censored survival data using binary predictor variables scenario indicate that the Survival Support Vector Machine (SSVM) method with Radial Basic Function Kernel (RBF Kernel) has the best model performance on data with small volumes, whereas when the data volume becomes larger, the method that has the best performance is Survival Support Vector Machine using Additive Kernel. Meanwhile, Random Survival Forest is a method that has the best performance for all conditions in mixed predictor variables scenario. Method, proportion of censored data and size of data are factors that affect the model performance.

The Restricted DINA Model: A Comprehensive Cognitive Diagnostic Model for Classroom-Level Assessments

The nonparametric classification (NPC) method has been proven to be a suitable procedure for cognitive diagnostic assessments at a classroom level. However, its nonparametric nature impedes the obtention of a model likelihood, hindering the exploration of crucial psychometric aspects, such as model fit or reliability. Reporting the reliability and validity of scores is imperative in any applied context. The present study proposes the restricted deterministic input, noisy “and” gate (R-DINA) model, a parametric cognitive diagnosis model based on the NPC method that provides the same attribute profile classifications as the nonparametric method while allowing to derive a model likelihood and, subsequently, to compute fit and reliability indices. The suitability of the new proposal is examined by means of an exhaustive simulation study and a real data illustration. The results show that the R-DINA model properly recovers the posterior probabilities of attribute mastery, thus becoming a suitable alternative for comprehensive small-scale diagnostic assessments.

A general nonparametric classification method for multiple strategies in cognitive diagnostic assessment.

Cognitive diagnosis models (CDMs) have been used as psychometric tools in educational assessments to estimate students' strengths and weaknesses in terms of cognitive skills learned and skills that need study. In practice, it is not uncommon that questions can often be solved using more than one strategy, which requires CDMs capable of accommodating multiple strategies. However, existing parametric multi-strategy CDMs need a large sample size to produce a reliable estimation of item parameters and examinees' proficiency class memberships, which obstructs their practical applications. This article proposes a general nonparametric multi-strategy classification method with promising classification accuracy in small samples for dichotomous response data. The method can accommodate different strategy selection approaches and different condensation rules. Simulation studies showed that the proposed method outperformed the parametric CDMs when sample sizes were small. A set of real data was analyzed as well to illustrate the application of the proposed method in practice.

Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data

In cognitive diagnosis assessment (CDA), the impact of misspecified item-attribute relations (or “Q-matrix”) designed by subject-matter experts has been a great challenge to real-world applications. This study examined parameter estimation of the CDA with the expert-designed Q-matrix and two refined Q-matrices for international large-scale data. Specifically, the G-DINA model was used to analyze TIMSS data for Grade 8 for five selected countries separately; and the need of a refined Q-matrix specific to the country was investigated. The results suggested that the two refined Q-matrices fitted the data better than the expert-designed Q-matrix, and the stepwise validation method performed better than the nonparametric classification method, resulting in a substantively different classification of students in attribute mastery patterns and different item parameter estimates. The results confirmed that the use of country-specific Q-matrices based on the G-DINA model led to a better fit compared to a universal expert-designed Q-matrix.

Sums of amplitudes analysis – A new non-parametric classification method for time series deviation evaluation in pharmaceutical processes

The evaluation of various types of data and the understanding of their properties is the backbone of scientific work. Time series data are abundant in the investigation of manufacturing processes. Since they result from measurements of parameters over time, time series have distinctive statistical characteristics. Focused on bulk solid dosing, the authors illustrate the potential for misinterpretations if these characteristics are ignored in data evaluation. The sums of amplitudes analysis (SAA) is developed, based on discrete Fourier transformations as a novel non-parametric statistical method for description of time series. Subsequently, the authors use the SAA to investigate bulk solid dosing data. In this process, artifications of data of external catch scales, commonly used for the evaluation of bulk solid dosing, are found, and discussed. • Quantification of the systematic information in pharmaceutics related time series. • Description of artefacts in powder dosing evaluations. • Fourier based non-parametric time series evaluation.

Supervised Nonparametric Classification in the Context of Replicated Point Patterns

A spatial point pattern is a collection of points in space, representing, e.g. observed locations of trees, bird nests, centers of cells in a histological sample, etc. When several independent realizations of the underlying stochastic process are observed, these realizations are referred to as replicated point patterns. The main objective of this paper is to classify a newly observed pattern into one of the existing classes using a supervised nonparametric classification method, namely the Bayes classifier in combination with the k-nearest neighbors algorithms and the kernel regression method. The dissimilarity between a pair of patterns is defined using the functional summaries extracted from the point patterns via the Cramér-von Mises or Kolmogorov-Smirnov type formula. A set of simulation experiments is presented to investigate the performance of the proposed classifier with a dissimilarity measure based on functional summaries, such as the pair correlation function. The application of such a classifier to a real point pattern dataset is also illustrated.

A New Fuzzy Adaptive Algorithm to Classify Imbalanced Data

Classification of imbalanced data is a well explored issue in the data mining and machine learning community where one class representation is overwhelmed by other classes. The Imbalanced distribution of data is a natural occurrence in real world datasets, so needed to be dealt with carefully to get important insights. In case of imbalance in data sets, traditional classifiers have to sacrifice their performances, therefore lead to misclassifications. This paper suggests a weighted nearest neighbor approach in a fuzzy manner to deal with this issue. We have adapted the ‘existing algorithm modification solution’ to learn from imbalanced datasets that classify data without manipulating the natural distribution of data unlike the other popular data balancing methods. The K nearest neighbor is a non-parametric classification method that is mostly used in machine learning problems. Fuzzy classification with the nearest neighbor clears the belonging of an instance to classes and optimal weights with improved nearest neighbor concept helping to correctly classify imbalanced data. The proposed hybrid approach takes care of imbalance nature of data and reduces the inaccuracies appear in applications of original and traditional classifiers. Results show that it performs well over the existing fuzzy nearest neighbor and weighted neighbor strategies for imbalanced learning.

An Adaptive Design for Item Parameter Online Estimation and Q-Matrix Online Calibration in CD-CAT.

The implementation of cognitive diagnostic computerized adaptive testing often depends on a high-quality item bank. How to online estimate the item parameters and calibrate the Q-matrix required by items becomes an important problem in the construction of the high-quality item bank for personalized adaptive learning. The related previous research mainly focused on the calibration method with the random design in which the new items were randomly assigned to examinees. Although the way of randomly assigning new items can ensure the randomness of data sampling, some examinees cannot provide enough information about item parameter estimation or Q-matrix calibration for the new items. In order to increase design efficiency, we investigated three adaptive designs under different practical situations: (a) because the non-parametric classification method needs calibrated item attribute vectors, but not item parameters, the first study focused on an optimal design for the calibration of the Q-matrix of the new items based on Shannon entropy; (b) if the Q-matrix of the new items was specified by subject experts, an optimal design was designed for the estimation of item parameters based on Fisher information; and (c) if the Q-matrix and item parameters are unknown for the new items, we developed a hybrid optimal design for simultaneously estimating them. The simulation results showed that, the adaptive designs are better than the random design with a limited number of examinees in terms of the correct recovery rate of attribute vectors and the precision of item parameters.

PERFORMANCE OF THE SUPPORT VECTOR MACHINE AND ARTIFICIAL NEURAL NETWORK CLASSIFIERS FOR ROADS IDENTIFICATION

Abstract. The objective of this project was to compare two non-parametric classification methods (“Support Vector Machine” – SVM and “Artificial Neural Networks” – ANN) of road regions in high spatial resolution images and associated with data from Airborne Laser Scanning. The study aims to verify what kind of influence the layers of attributes have on the performance from respective classifiers: SVM and RNA. Our method based on tests of this classifiers on 4 bands of airborne images and normalization of the digital surface model (DSM) for showing only information on objects height in relation to ground and not of these in relation to the ground and relief, generating band 5. The samples were used to train chosen non-parametric classifiers (training sets for each different input image/landscape). All classifications had the same set of training samples and the same classification parameters. The optimal parameters for classifications were obtained through the existing library in the Weka mining package: LibSVM and LibMultiLayerPerceptron. Our results demonstrated the existence of a direct relationship between the elevation band of the targets in relation to the terrain (band 05) with the improvement of their performance and lower degree of between bands correlation can also be considered a factor that has a positive influence. As for Neural Networks, the experiment results demonstrate that the presence of the near infrared band (band 04) was decisive for the performance improving of certain combinations in relation to others.

Semi-Automatic Method for Early Detection of Xylella fastidiosa in Olive Trees Using UAV Multispectral Imagery and Geostatistical-Discriminant Analysis

Xylella fastidiosa subsp. pauca (Xfp) is one of the most dangerous plant pathogens in the world. Identified in 2013 in olive trees in south–eastern Italy, it is spreading to the Mediterranean countries. The bacterium is transmitted by insects that feed on sap, and causes rapid wilting in olive trees. The paper explores the use of Unmanned Aerial Vehicle (UAV) in combination with a multispectral radiometer for early detection of infection. The study was carried out in three olive groves in the Apulia region (Italy) and involved four drone flights from 2017 to 2019. To classify Xfp severity level in olive trees at an early stage, a combined method of geostatistics and discriminant analysis was implemented. The results of cross-validation for the non-parametric classification method were of overall accuracy = 0.69, mean error rate = 0.31, and for the early detection class of accuracy 0.77 and misclassification probability 0.23. The results are promising and encourage the application of UAV technology for the early detection of Xfp infection.

Data Integration by Combining Big Data and Survey Sample Data for Finite Population Inference

SummaryThe statistical challenges in using big data for making valid statistical inference in the finite population have been well documented in literature. These challenges are due primarily to statistical bias arising from under‐coverage in the big data source to represent the population of interest and measurement errors in the variables available in the data set. By stratifying the population into a big data stratum and a missing data stratum, we can estimate the missing data stratum by using a fully responding probability sample and hence the population as a whole by using a data integration estimator. By expressing the data integration estimator as a regression estimator, we can handle measurement errors in the variables in big data and also in the probability sample. We also propose a fully nonparametric classification method for identifying the overlapping units and develop a bias‐corrected data integration estimator under misclassification errors. Finally, we develop a two‐step regression data integration estimator to deal with measurement errors in the probability sample. An advantage of the approach advocated in this paper is that we do not have to make unrealistic missing‐at‐random assumptions for the methods to work. The proposed method is applied to the real data example using 2015–2016 Australian Agricultural Census data.

Fuel Type Classification Using Airborne Laser Scanning and Sentinel 2 Data in Mediterranean Forest Affected by Wildfires

Mediterranean forests are recurrently affected by fire. The recurrence of fire in such environments and the number and severity of previous fire events are directly related to fire risk. Fuel type classification is crucial for estimating ignition and fire propagation for sustainable forest management of these wildfire prone environments. The aim of this study is to classify fuel types according to Prometheus classification using low-density Airborne Laser Scanner (ALS) data, Sentinel 2 data, and 136 field plots used as ground-truth. The study encompassed three different Mediterranean forests dominated by pines (Pinus halepensis, P. pinaster y P. nigra), oaks (Quercus ilex) and quercus (Q. faginea) in areas affected by wildfires in 1994 and their surroundings. Two metric selection approaches and two non-parametric classification methods with variants were compared to classify fuel types. The best-fitted classification model was obtained using Support Vector Machine method with radial kernel. The model includes three ALS and one Sentinel-2 metrics: the 25th percentile of returns height, the percentage of all returns above mean, rumple structural diversity index and NDVI. The overall accuracy of the model after validation was 59%. The combination of data from active and passive remote sensing sensors as well as the use of adapted structural diversity indices derived from ALS data improved accuracy classification. This approach demonstrates its value for mapping fuel type spatial patterns at a regional scale under different heterogeneous and topographically complex Mediterranean forests.

Gaussian Mixture Model Based Classification Revisited: Application to the Bearing Fault Classification

Condition monitoring and fault detection are nowadays popular topic. Different loads, enviroments etc. affect the components and systems differently and can induce the fault and faulty behaviour. Most of the approaches for the fault detection rely on the use of the good classification method. Gaussian mixture model based classification are stable and versatile methods which can be applied to a wide range of classification tasks. The main task is the estimation of the parameters in the Gaussian mixture model. Those can be estimated with various techniques. Therefore, the Gaussian mixture model based classification have different variants which can vary in performance. To test the performance of the Gaussian mixture model based classification variants and general usefulness of the Gaussian mixture model based classification for the fault detection, we have opted to use the bearing fault classification problem. Additionally, comparisons with other widely used non-parametric classification methods are made, such as support vector machines and neural networks. The performance of each classification method is evaluated by multiple repeated k-fold cross validation. From the results obtained, Gaussian mixture model based classification methods are shown to be competitive and efficient methods and usable in the field of fault detection and condition monitoring.

Referandum Sonucunun Seçmenlerin Yerel Yönetime Tutumu İle Tahmin Edilmesi

In recent years, the referendums play an important role in modern democracies. With the rapid changes in globalization, information and communication technologies and the increasing complexity of economic and social problems caused by intense competition, especially the referendums that are held to maximize the legitimacy of the decisions taken by the public will are seen as direct participation of citizens in the political process when traditional forms of political participation are declining. This study present a method to predict how citizen will vote in the referendum only by starting from their attitude towards local governments. For this purpose, a qualitative research was designed and 2446 voters were interviewed in Kilis Province Center of Turkey. In the study, Support Vector Machines, which is a nonparametric classification method based on statistical learning theory, was used to estimate the referendum results that would be voted as yes or no. At the end of the study, by using twenty-six variables, the rate of people voting in the referendum was predicted by 81%. This results guide the survey companies' efforts to predict the results of the referendum

A New K-Nearest Neighbors Classifier for Big Data Based on Efficient Data Pruning

The K-nearest neighbors (KNN) machine learning algorithm is a well-known non-parametric classification method. However, like other traditional data mining methods, applying it on big data comes with computational challenges. Indeed, KNN determines the class of a new sample based on the class of its nearest neighbors; however, identifying the neighbors in a large amount of data imposes a large computational cost so that it is no longer applicable by a single computing machine. One of the proposed techniques to make classification methods applicable on large datasets is pruning. LC-KNN is an improved KNN method which first clusters the data into some smaller partitions using the K-means clustering method; and then applies the KNN for each new sample on the partition which its center is the nearest one. However, because the clusters have different shapes and densities, selection of the appropriate cluster is a challenge. In this paper, an approach has been proposed to improve the pruning phase of the LC-KNN method by taking into account these factors. The proposed approach helps to choose a more appropriate cluster of data for looking for the neighbors, thus, increasing the classification accuracy. The performance of the proposed approach is evaluated on different real datasets. The experimental results show the effectiveness of the proposed approach and its higher classification accuracy and lower time cost in comparison to other recent relevant methods.

Basic Tenets of Classification Algorithms K-Nearest-Neighbor, Support Vector Machine, Random Forest and Neural Network: A Review

In this paper, sixty-eight research articles published between 2000 and 2017 as well as textbooks which employed four classification algorithms: K-Nearest-Neighbor (KNN), Support Vector Machines (SVM), Random Forest (RF) and Neural Network (NN) as the main statistical tools were reviewed. The aim was to examine and compare these nonparametric classification methods on the following attributes: robustness to training data, sensitivity to changes, data fitting, stability, ability to handle large data sizes, sensitivity to noise, time invested in parameter tuning, and accuracy. The performances, strengths and shortcomings of each of the algorithms were examined, and finally, a conclusion was arrived at on which one has higher performance. It was evident from the literature reviewed that RF is too sensitive to small changes in the training dataset and is occasionally unstable and tends to overfit in the model. KNN is easy to implement and understand but has a major drawback of becoming significantly slow as the size of the data in use grows, while the ideal value of K for the KNN classifier is difficult to set. SVM and RF are insensitive to noise or overtraining, which shows their ability in dealing with unbalanced data. Larger input datasets will lengthen classification times for NN and KNN more than for SVM and RF. Among these nonparametric classification methods, NN has the potential to become a more widely used classification algorithm, but because of their time-consuming parameter tuning procedure, high level of complexity in computational processing, the numerous types of NN architectures to choose from and the high number of algorithms used for training, most researchers recommend SVM and RF as easier and wieldy used methods which repeatedly achieve results with high accuracies and are often faster to implement.

Multi-group diagnostic classification of high-dimensional data using differential scanning calorimetry plasma thermograms.

The thermoanalytical technique differential scanning calorimetry (DSC) has been applied to characterize protein denaturation patterns (thermograms) in blood plasma samples and relate these to a subject’s health status. The analysis and classification of thermograms is challenging because of the high-dimensionality of the dataset. There are various methods for group classification using high-dimensional data sets; however, the impact of using high-dimensional data sets for cancer classification has been poorly understood. In the present article, we proposed a statistical approach for data reduction and a parametric method (PM) for modeling of high-dimensional data sets for two- and three- group classification using DSC and demographic data. We compared the PM to the non-parametric classification method K-nearest neighbors (KNN) and the semi-parametric classification method KNN with dynamic time warping (DTW). We evaluated the performance of these methods for multiple two-group classifications: (i) normal versus cervical cancer, (ii) normal versus lung cancer, (iii) normal versus cancer (cervical + lung), (iv) lung cancer versus cervical cancer as well as for three-group classification: normal versus cervical cancer versus lung cancer. In general, performance for two-group classification was high whereas three-group classification was more challenging, with all three methods predicting normal samples more accurately than cancer samples. Moreover, specificity of the PM method was mostly higher or the same as KNN and DTW-KNN with lower sensitivity. The performance of KNN and DTW-KNN decreased with the inclusion of demographic data, whereas similar performance was observed for the PM which could be explained by the fact that the PM uses fewer parameters as compared to KNN and DTW-KNN methods and is thus less susceptible to the risk of overfitting. More importantly the accuracy of the PM can be increased by using a greater number of quantile data points and by the inclusion of additional demographic and clinical data, providing a substantial advantage over KNN and DTW-KNN methods.

ANALISIS PERBANDINGAN KINERJA CART KONVENSIONAL, BAGGING DAN RANDOM FOREST PADA KLASIFIKASI OBJEK: HASIL DARI DUA SIMULASI

The conventional CART method is a nonparametric classification method built on categorical response data. Bagging is one of the popular ensemble methods whereas, Random Forests (RF) is one of the relatively new ensemble methods in the decision tree that is the development of the Bagging method. Unlike Bagging, Random Forest was developed with the idea of adding layers to the random resampling process in Bagging. Therefore, not only randomly sampled sample data to form a classification tree, but also independent variables are randomly selected and newly selected as the best divider when determining the sorting of trees, which is expected to produce more accurate predictions. Based on the above, the authors are interested to study the three methods by comparing the accuracy of classification on binary and non-binary simulation data to understand the effect of the number of sample sizes, the correlation between independent variables, the presence or absence of certain distribution patterns to the accuracy generated classification method. results of the research on simulation data show that the Random Forest ensemble method can improve the accuracy of classification.

The k conditional nearest neighbor algorithm for classification and class probability estimation.

The k nearest neighbor (kNN) approach is a simple and effective nonparametric algorithm for classification. One of the drawbacks of kNN is that the method can only give coarse estimates of class probabilities, particularly for low values of k. To avoid this drawback, we propose a new nonparametric classification method based on nearest neighbors conditional on each class: the proposed approach calculates the distance between a new instance and the kth nearest neighbor from each class, estimates posterior probabilities of class memberships using the distances, and assigns the instance to the class with the largest posterior. We prove that the proposed approach converges to the Bayes classifier as the size of the training data increases. Further, we extend the proposed approach to an ensemble method. Experiments on benchmark data sets show that both the proposed approach and the ensemble version of the proposed approach on average outperform kNN, weighted kNN, probabilistic kNN and two similar algorithms (LMkNN and MLM-kHNN) in terms of the error rate. A simulation shows that kCNN may be useful for estimating posterior probabilities when the class distributions overlap.

Compressed kNN: K-Nearest Neighbors with Data Compression.

The kNN (k-nearest neighbors) classification algorithm is one of the most widely used non-parametric classification methods, however it is limited due to memory consumption related to the size of the dataset, which makes them impractical to apply to large volumes of data. Variations of this method have been proposed, such as condensed KNN which divides the training dataset into clusters to be classified, other variations reduce the input dataset in order to apply the algorithm. This paper presents a variation of the kNN algorithm, of the type structure less NN, to work with categorical data. Categorical data, due to their nature, can be compressed in order to decrease the memory requirements at the time of executing the classification. The method proposes a previous phase of compression of the data to then apply the algorithm on the compressed data. This allows us to maintain the whole dataset in memory which leads to a considerable reduction of the amount of memory required. Experiments and tests carried out on known datasets show the reduction in the volume of information stored in memory and maintain the accuracy of the classification. They also show a slight decrease in processing time because the information is decompressed in real time (on-the-fly) while the algorithm is running.