Non-parametric Classification Technique Research Articles

Bayesian neural tree models for nonparametric regression

SummaryFrequentist and Bayesian methods differ in many aspects but share some basic optimal properties. In real‐life prediction problems, situations exist in which a model based on one of the above paradigms is preferable depending on some subjective criteria. Nonparametric classification and regression techniques, such as decision trees and neural networks, have both frequentist (classification and regression trees (CARTs) and artificial neural networks) as well as Bayesian counterparts (Bayesian CART and Bayesian neural networks) to learning from data. In this paper, we present two hybrid models combining the Bayesian and frequentist versions of CART and neural networks, which we call the Bayesian neural tree (BNT) models. BNT models can simultaneously perform feature selection and prediction, are highly flexible, and generalise well in settings with limited training observations. We study the statistical consistency of the proposed approaches and derive the optimal value of a vital model parameter. The excellent performance of the newly proposed BNT models is shown using simulation studies. We also provide some illustrative examples using a wide variety of standard regression datasets from a public available machine learning repository to show the superiority of the proposed models in comparison to popularly used Bayesian CART and Bayesian neural network models.

Klasifikasi Sekolah dalam Penerimaan Mahasiswa Baru Vokasi IPB Jalur USMI Menggunakan Metode CART

The selection of new student admissions for the IPB vocational school consists of several routes, one of which is the USMI route. In order to improve its performance, it is necessary to evaluate the USMI new student admission system. Previously, research with the same objective had been carried out using the clustering method. The study resulted in three clusters in which schools were differentiated based on commitment and quality. This study aims to create a classification model of the clusters obtained using the CART method. Classification and Regression Tree (CART) is a nonparametric classification technique that produces a single decision tree. The CART method can involve mixed-type data. The classification model generated from the 2019 data yields an accuracy of 98.52%. However, the results of 2019 model evaluation with the 2020 data are still not good enough to predict with an accuracy of 57.22%, so the 2020 data is re-clustered and produces three clusters. Furthermore, the classification model was remade with 2020 data, resulting in an accuracy of 97.47%. However, the results of the 2020 model evaluation with the 2021 data are still not good enough to predict with an accuracy of 44.34%, so the clustering in the previous year cannot be used for predictions of the following year's data. The grouping of schools for USMI applicants needs to be done by grouping schools every year.

A two-step machine learning approach to predict S&P 500 bubbles

ABSTRACT In this paper, we are interested in predicting the bubbles in the S&P 500 stock market with a two-step machine learning approach that employs a real-time bubble detection test and support vector machine (SVM). SVM as a nonparametric binary classification technique is already a widely used method in financial time series forecasting. In the literature, a bubble is often defined as a situation where the asset price exceeds its fundamental value. As one of the early warning signals, prediction of bubbles is vital for policymakers and regulators who are responsible to take preemptive measures against the future crises. Therefore, many attempts have been made to understand the main factors in bubble formation and to predict them in their earlier phases. Our analysis consists of two steps. The first step is to identify the bubbles in the S&P 500 index using a widely recognized right-tailed unit root test. Then, SVM is employed to predict the bubbles by macroeconomic indicators. Also, we compare SVM with different supervised learning algorithms by using k-fold cross-validation. The experimental results show that the proposed approach with high predictive power could be a favourable alternative in bubble prediction.

An online isotonic separation with cascade architecture for binary classification

Isotonic separation (IS) is a non-parametric classification technique which constructs an isotonic function from ordered data. The rationale is to convert partially isotonic data into isotonic using a linear programming problem (LPP) and partition the input space into isotonic and non-isotonic regions to make predictions easier. Despite the widespread applications of IS in diverse domains where monotonicity exists, it has certain limitations: Firstly, computing time and the constraints of the LPP in isotonic separation increase polynomially as size of the data increases and it is highly complex to solve the LPP and obtain the model on large data sets. In order to support dynamic stream data and address the computational overhead and size of the LPP issues, this paper proposes an online isotonic separation algorithm called Cascade-IS (CIS) for binary classification. The rationale behind CIS is that it splits the data set into a sequence of partitions and models are obtained and combined in cascade. Statistical and experimental analysis are done on datasets with isotonic properties and the results prove that CIS is superior to its variants in terms of training time, performance measures and number of constraints in the LPP.

Investigation of helmet use behavior of motorcyclists and effectiveness of enforcement campaign using CART approach

Use of the helmet is beneficial in reducing the severity of injuries and avoiding fatalities for motorcyclists, therefore, legislation exists in almost all countries. In practice, the situation is different regarding the helmet use, especially in the developing countries. This paper investigates the helmet use behavior of motorcyclists in Karachi, Pakistan. It determines the significant factors affecting the helmet use in Karachi (Pakistan) and recommending effective campaign measures to promote helmet use. It is vital in relation to the exponential growth of motorcycle users and poor enforcement of traffic rules. Repeated cross-sectional data collected before (n = 226) and after (n = 277) the helmet use enforcement campaign is analyzed using univariate and non-parametric classification and regression tree (CART) techniques. A significant number of motorcyclists do not hold a driving license and CART analysis highlighted the significance of this variable along with exposure to the road environment (measured as daily usage of motorcycle) to explain helmet use. Campaign effects are found temporary, therefore, serious efforts are required to design coherent and structured awareness and enforcement programs.

A new non-parametric classifier to predict small-business failures in Italy via performance ratios

We considered the case of small-medium enterprises (SMEs) in Italy introducing a new classifier to predict bankruptcy up to eight years prior to failure. We considered a stratified random sample of 100 non-listed Italian SMEs, 50 of which filed for bankruptcy during the years 2000 to 2011. Results suggest that the proposed method more than holds its own when compared with standard non-parametric classification techniques. The performance of the proposed method based on recognition rate, sensitivity and specificity shows that the proposed technique is effective in predicting the failure of a firm up to eight years prior to the event. The high specificity makes the proposed technique very effective as a warning signal to determine if a firm is in distress with a sufficient enough time to take proper actions. The performance assessment has been achieved via cross-validation to get unbiased estimates of the performances.

Energy-Arrival-Aware Detection Threshold in Wireless-Powered Cognitive Radio Networks

Traffic patterns associated with different primary user (PU) channels may provide different spectral access and energy harvesting opportunities in wireless-powered cognitive radio networks (WP-CRNs). Considering this, we propose traffic-specific optimal spectrum sensing policy such that the expected transmission rate of secondary user (SU) is maximized under the energy causality and PU collision constraints in orthogonal frequency-division multiple access (OFDMA)-based WP-CRN. Toward this, we cluster $N$ subcarriers (subchannels) into $K$ clusters (where $K \leq N$ ) and derive an optimal energy detection threshold for the SU under each traffic pattern. Using traffic features, we propose an unsupervised and nonparametric classification technique to determine the number of unique traffic patterns $K$ over all subchannels. Then, the traffic patterns are used to predict the idle/busy period statistics for subchannels, based upon which SU identifies harvest and transmit PU subchannels for energy harvesting and data transmission, respectively. We derive an optimal detection threshold based on the harvested energy such that it maximizes the expected transmission rate of SU while protecting PU from collision. We demonstrate the effectiveness of the proposed scheme in terms of rate gains under design constraints and show the optimal detection threshold under various energy arrival rates.

Re-examining the middle-income trap hypothesis (MITH): What to reject and what to revive?

Do middle-income countries face difficult challenges producing consistent growth? Using transition matrix analysis, we can easily reject any unconditional notion of a “middle-income trap” in the data. However, countries have different fundamentals and policies. Using a non-parametric classification technique, we search for variables that separate fast- and slow-growing countries. For middle-income countries, a relatively large working age population, sex ratio imbalance, macroeconomic stability, and financial development appear to be the key discriminatory variables. We do the same exercise for low-income countries. This framework yields conditions under which countries in the low- and middle-income ranges move forward or backward, or are trapped.

Evaluating the reliability of field identification and morphometric classifications for carnivore scats confirmed with genetic analysis

ABSTRACTScat surveys are commonly used to monitor carnivore populations. Scats of sympatric carnivores can be difficult to differentiate and field‐based identification can be misleading. We evaluated the success of field‐based species identification for scats of 2 sympatric carnivores—coyotes (Canis latrans) and kit foxes (Vulpes macrotis). We conducted scat surveys in the Great Basin desert of Utah, USA, during the winter and summer of 2013, and we detected 1,680 carnivore scats. We classified scats based on field identification, recorded morphometric measurements, and collected fecal DNA samples for molecular species identification. We subsequently evaluated the classification success of field identification and the predictive power of 2 nonparametric classification techniques—k‐nearest neighbors and classification trees—based on scat measurements. Overall, 12.2% of scats were misclassified by field identification, but misclassifications were not equitable between species. Only 7.1% of the scats identified as coyote with field identification were misclassified, compared with 22.9% of scats identified as kit fox. Results from both k‐nearest neighbor and classification‐tree analyses suggest that morphometric measurements provided an objective alternative to field identification that improved classification of rarer species. Overall misclassification rates for k‐nearest neighbor and classification‐tree analyses were 11.7% and 7.5%, respectively. Using classification trees, misclassification was reduced for kit foxes (8.5%) and remained similar for coyotes (7.2%), relative to field identification. Although molecular techniques provide unambiguous species identification, classification approaches may offer a cost‐effective alternative. We recommend that monitoring programs employing scat surveys utilize molecular species identification to develop training data sets and evaluate the accuracy of field‐based and statistical classification approaches. © 2015 The Wildlife Society.

A general proof of consistency of heuristic classification for cognitive diagnosis models

The Asymptotic Classification Theory of Cognitive Diagnosis (Chiu et al., 2009, Psychometrika, 74, 633-665) determined the conditions that cognitive diagnosis models must satisfy so that the correct assignment of examinees to proficiency classes is guaranteed when non-parametric classification methods are used. These conditions have only been proven for the Deterministic Input Noisy Output AND gate model. For other cognitive diagnosis models, no theoretical legitimization exists for using non-parametric classification techniques for assigning examinees to proficiency classes. The specific statistical properties of different cognitive diagnosis models require tailored proofs of the conditions of the Asymptotic Classification Theory of Cognitive Diagnosis for each individual model – a tedious undertaking in light of the numerous models presented in the literature. In this paper a different way is presented to address this task. The unified mathematical framework of general cognitive diagnosis models is used as a theoretical basis for a general proof that under mild regularity conditions any cognitive diagnosis model is covered by the Asymptotic Classification Theory of Cognitive Diagnosis.

Generalized quadratic discriminant analysis

In linear discriminant analysis, the assumption of equality of the dispersion matrices of different classes leads to a classification rule based on minimum Mahalanobis distance from the class centres. However, without this assumption, the resulting quadratic discriminant classifier involves, in addition to the Mahalanobis distances, the ratio of the determinants of the dispersion matrices as a factor. In fact, it has been observed that, for discriminating between populations with underlying elliptically symmetric distributions, such classifiers also incorporate similar factors, apart from the Mahalanobis distances.In this paper, a nonparametric classification technique which generalizes discriminant analysis has been proposed. The method of cross-validation is used to make the technique adaptive to a given dataset. An extensive simulation study is presented to illustrate the potential of the method. Finally, through implementation on a number of real-life data sets, it has been demonstrated that the proposed generalized quadratic discriminant analysis (GQDA) compares very favourably with other nonparametric methods, and is computationally cost-effective.

Re-Examining the Middle-Income Trap Hypothesis: What to Reject and What to Revive?

Do middle-income countries face difficult challenges producing consistent growth? Using transition matrix analysis, we can easily reject any unconditional notion of a 'middle-income trap' in the data. However, countries have different fundamentals and policies. Using a non-parametric classification technique, we search for variables that separate fast- and slow-growing countries. For middle-income countries, a relatively large working age population, sex ratio imbalance, macroeconomic stability, and financial development appear to be the key discriminatory variables. We do the same exercise for low-income countries. This framework yields conditions under which countries in the low- and middle-income ranges are trapped or even move backward.

Heuristic cognitive diagnosis when the Q-matrix is unknown.

Cognitive diagnosis models of educational test performance rely on a binary Q-matrix that specifies the associations between individual test items and the cognitive attributes (skills) required to answer those items correctly. Current methods for fitting cognitive diagnosis models to educational test data and assigning examinees to proficiency classes are based on parametric estimation methods such as expectation maximization (EM) and Markov chain Monte Carlo (MCMC) that frequently encounter difficulties in practical applications. In response to these difficulties, non-parametric classification techniques (cluster analysis) have been proposed as heuristic alternatives to parametric procedures. These non-parametric classification techniques first aggregate each examinee's test item scores into a profile of attribute sum scores, which then serve as the basis for clustering examinees into proficiency classes. Like the parametric procedures, the non-parametric classification techniques require that the Q-matrix underlying a given test be known. Unfortunately, in practice, the Q-matrix for most tests is not known and must be estimated to specify the associations between items and attributes, risking a misspecified Q-matrix that may then result in the incorrect classification of examinees. This paper demonstrates that clustering examinees into proficiency classes based on their item scores rather than on their attribute sum-score profiles does not require knowledge of the Q-matrix, and results in a more accurate classification of examinees.

AncesTrees: ancestry estimation with randomized decision trees.

In forensic anthropology, ancestry estimation is essential in establishing the individual biological profile. The aim of this study is to present a new program--AncesTrees--developed for assessing ancestry based on metric analysis. AncesTrees relies on a machine learning ensemble algorithm, random forest, to classify the human skull. In the ensemble learning paradigm, several models are generated and co-jointly used to arrive at the final decision. The random forest algorithm creates ensembles of decision trees classifiers, a non-linear and non-parametric classification technique. The database used in AncesTrees is composed by 23 craniometric variables from 1,734 individuals, representative of six major ancestral groups and selected from the Howells' craniometric series. The program was tested in 128 adult crania from the following collections: the African slaves' skeletal collection of Valle da Gafaria; the Medical School Skull Collection and the Identified Skeletal Collection of 21st Century, both curated at the University of Coimbra. The first step of the test analysis was to perform ancestry estimation including all the ancestral groups of the database. The second stage of our test analysis was to conduct ancestry estimation including only the European and the African ancestral groups. In the first test analysis, 75% of the individuals of African ancestry and 79.2% of the individuals of European ancestry were correctly identified. The model involving only African and European ancestral groups had a better performance: 93.8% of all individuals were correctly classified. The obtained results show that AncesTrees can be a valuable tool in forensic anthropology.

Binary classification of chalcone derivatives with LDA or KNN based on their antileishmanial activity and molecular descriptors selected using the Successive Projections Algorithm feature-selection technique

Chalcones are naturally occurring aromatic ketones, which consist of an α-, β-unsaturated carbonyl system joining two aryl rings. These compounds are reported to exhibit several pharmacological activities, including antiparasitic, antibacterial, antifungal, anticancer, immunomodulatory, nitric oxide inhibition and anti-inflammatory effects. In the present work, a Quantitative Structure–Activity Relationship (QSAR) study is carried out to classify chalcone derivatives with respect to their antileishmanial activity (active/inactive) on the basis of molecular descriptors. For this purpose, two techniques to select descriptors are employed, the Successive Projections Algorithm (SPA) and the Genetic Algorithm (GA). The selected descriptors are initially employed to build Linear Discriminant Analysis (LDA) models. An additional investigation is then carried out to determine whether the results can be improved by using a non-parametric classification technique (One Nearest Neighbour, 1NN). In a case study involving 100 chalcone derivatives, the 1NN models were found to provide better rates of correct classification than LDA, both in the training and test sets. The best result was achieved by a SPA–1NN model with six molecular descriptors, which provided correct classification rates of 97% and 84% for the training and test sets, respectively.

A Study of Student Satisfaction in the University of Agriculture Faisalabad

The outcome variable for student's satisfaction was measured on five points likert-scale; Strongly Disagree, Disagree, No Response, Agree and Strongly Agree. The questionnaire under study was consisted of gender, faculty and 27 explanatory variables related to the satisfaction of students about the overall university experiences. To measure the student's satisfaction is an interesting work for the university's policy makers to enhance the quality of services and opportunities for students. A parsimonious model was constructed by applying ordinal logistic regression technique to see the relationship among the ordinal outcome response variables and explanatory variables. Statistically the focus of this study was to compare the parametric and non-parametric classification techniques.

Interpretable support vector machines for functional data

Support Vector Machines (SVMs) is known to be a powerful nonparametric classification technique even for high-dimensional data. Although predictive ability is important, obtaining an easy-to-interpret classifier is also crucial in many applications. Linear SVM provides a classifier based on a linear score. In the case of functional data, the coefficient function that defines such linear score usually has many irregular oscillations, making it difficult to interpret.This paper presents a new method, called Interpretable Support Vector Machines for Functional Data, that provides an interpretable classifier with high predictive power. Interpretability might be understood in different ways. The proposed method is flexible enough to cope with different notions of interpretability chosen by the user, thus the obtained coefficient function can be sparse, linear-wise, smooth, etc. The usefulness of the proposed method is shown in real applications getting interpretable classifiers with comparable, sometimes better, predictive ability versus classical SVM.

Classification of Periodic Activities Using the Wasserstein Distance

In this paper, we introduce a novel nonparametric classification technique based on the use of the Wasserstein distance. The proposed scheme is applied in a biomedical context for the analysis of recorded accelerometer data: the aim is to retrieve three types of periodic activities (walking, biking, and running) from a time-frequency representation of the data. The main interest of the use of the Wasserstein distance lies in the fact that it is less sensitive to the location of the frequency peaks than to the global structure of the frequency pattern, allowing us to detect activities almost independently of their speed or incline. Our system is tested on a 24-subject corpus: results show that the use of Wasserstein distance combined with some supervised learning techniques allows us to compare with some more complex classification systems.

Advanced Time–Frequency Mutual Information Measures for Condition-Based Maintenance of Helicopter Drivetrains

A new concept of nonparametric signal detection and classification technique is proposed using mutual information measures in the time-frequency domain. The time-frequency-based self-information and mutual information are defined in terms of the cross time-frequency distribution. Based on time-frequency mutual information theory, this paper presents applications of the proposed technique to real-world vibration data obtained from a dedicated condition-based-maintenance experimental test bed. Baseline, unbalanced, and misaligned experimental settings of helicopter drivetrain bearings and shafts are quantitatively distinguished by the proposed techniques. With imbalance quantifiable by variance in the in-phase mutual information and misalignment quantifiable by variance in the quadrature mutual information developed and presented herein, machine health classification can be accomplished by use of statistical bounding regions.

Variable-length haplotype construction for geneߝgene interaction studies

This paper presents a non-parametric classification technique for identifying a candidate bi-allelic genetic marker set that best describes disease susceptibility in gene-gene interaction studies. The developed technique functions by creating a mapping between inferred haplotypes and case/control status. The technique cycles through all possible marker combination models generated from the available marker set where the best interaction model is determined from prediction accuracy and two auxiliary criteria including low-to-high order haplotype propagation capability and model parsimony. Since variable-length haplotypes are created during the best model identification, the developed technique is referred to as a variable-length haplotype construction for gene-gene interaction (VarHAP) technique. VarHAP has been benchmarked against a multifactor dimensionality reduction (MDR) program and a haplotype interaction technique embedded in a FAMHAP program in various two-locus interaction problems. The results reveal that VarHAP is suitable for all interaction situations with the presence of weak and strong linkage disequilibrium among genetic markers.