Non-parametric Statistical Learning Research Articles

Mapping the Expansion of Boom Crops in Mainland Southeast Asia Using Dense Time Stacks of Landsat Data

We performed a multi-date composite change detection technique using a dense-time stack of Landsat data to map land-use and land-cover change (LCLUC) in Mainland Southeast Asia (MSEA) with a focus on the expansion of boom crops, primarily tree crops. The supervised classification was performed using Support Vector Machines (SVM), which are supervised non-parametric statistical learning techniques. To select the most suitable SMV classifier and the related parameter settings, we used the training data and performed a two-dimensional grid search with a three-fold internal cross-validation. We worked in seven Landsat footprints and found the linear kernel to be the most suitable for all footprints, but the most suitable regularization parameter C varied across the footprints. We distinguished a total of 41 LCLUCs (13 to 31 classes per footprint) in very dynamic and heterogeneous landscapes. The approach proved useful for distinguishing subtle changes over time and to map a variety of land covers, tree crops, and transformations as long as sufficient training points could be collected for each class. While to date, this approach has only been applied to mapping urban extent and expansion, this study shows that it is also useful for mapping change in rural settings, especially when images from phenologically relevant acquisition dates are included.

Interactive and nonparametric modeling of preferences on an ordinal scale using small data

In this study, we consider learning preference structure of a Decision Maker (DM). Many preference modeling problems in a variety of fields such as marketing, quality control and economics involve possibly interacting criteria, and an ordinal scale is used to express preference of objects. In these cases, typically underlying preference structure of the DM and distribution of criteria values are not known, and only a few data can be collected about the preferences of the DM.For developing a preference model under such circumstances, we propose using nonparametric Statistical Learning approaches interactively. In particular, we employ Active Learning by asking a preference question to the DM at each step and try to reach a close approximation to the correct model in a small number of steps. Our experimental analysis proves that the proposed approach outperforms a “naive” approach where subsequent questions are asked randomly. In the study, we also provide algorithmic recommendations for modeling different underlying value functions, if information is available about the form of the preference structure and/or distribution of criteria values.This study can be regarded as a pioneering approach considering that Statistical Learning based approaches in the literature have been developed and tested based on a relatively large preference information and they do not interact with the DM in model developing process while Multi Criteria Decision Aid based approaches typically ignore interactions among the criteria, suffer from generalization ability, and have no concern about predicting equally good everywhere in the criteria domain.

Biologic Data of Cynomolgus Monkeys Maintained under Laboratory Conditions.

The cynomolgus monkey (Macaca fascicularis) is a well-known non-human primate species commonly used in non-clinical research. It is important to know basal clinical pathology parameters in order to have a reference for evaluating any potential treatment-induced effects, maintaining health status among animals and, if needed, evaluating correct substantiative therapies. In this study, data from 238 untreated cynomolgus monkeys (119 males and 119 females of juvenile age, 2.5 to 3.5 years) kept under laboratory conditions were used to build up a reference database of clinical pathology parameters. Twenty-two hematology markers, 24 clinical chemistry markers and two blood coagulation parameters were analyzed. Gender-related differences were evaluated using statistical analyses. To assess the possible effects of stress induced by housing or handling involved in treatment procedures, 78 animals (35 males and 35 females out of 238 juvenile monkeys and four adult males and four adult females) were used to evaluate cortisol, corticosterone and behavioral assessment over time. Data were analyzed using a non-parametric statistical test and machine learning approaches. Reference clinical pathology data obtained from untreated animals may be extremely useful for investigators employing cynomolgus monkeys as a test system for non-clinical safety studies.

Prediction with missing data via Bayesian Additive Regression Trees

AbstractWe present a method for incorporating missing data into general prediction problems which use nonparametric statistical learning. We focus on a tree‐based method, Bayesian Additive Regression Trees (BART), enhanced with “Missingness Incorporated in Attributes,” a recently proposed approach for incorporating missingness into decision trees. This procedure extends the native partitioning mechanisms found in tree‐based models and does not require imputation. Simulations on generated models and real data indicate that our procedure offers promise for both selection model and pattern‐mixture frameworks as measured by out‐of‐sample predictive accuracy. We also illustrate BART's abilities to incorporate missingness into uncertainty intervals. Our implementation is readily available in the R package bartMachine. The Canadian Journal of Statistics 43: 224–239; 2015 © 2015 Statistical Society of Canada

Nonparametric statistical learning control of robot manipulators for trajectory or contour tracking

This paper presents a method of precision tracking control for industrial robot manipulators. For robotic laser and plasma cutting tasks, the required tracking performance is much more demanding than that for material handling, spot welding, and machine tending tasks. Challenges in control come from the nonlinear coupled multi-body dynamics of robot manipulators, as well as the transmission error in the geared joints. The proposed method features data-driven iterative compensation of torque and motor reference. Motor side tracking and transmission error are handled by separate learning modules in a two-part compensation structure. Depending on the specific setup of end-effector sensing, the method can utilize either timed trajectory measurement or untimed two-dimensional contour inspection. Nonparametric statistical learning is used for the compensation. Considerations on incorporating analytical models and selecting data subsets for more efficient learning are discussed. The method is validated using a six-axis industrial robot. HighlightsA control method of robot manipulators for precision trajectory or contour tracking is presented.Nonparametric statistical learning is used for data-driven compensation of both torque and motor reference.A two-part compensation structure is adopted to handle motor side tracking and transmission error.The compensation can be done using either timed trajectory measurement or untimed contour inspection.

Application of Support Vector Machines for Landuse Classification Using High-Resolution RapidEye Images: A Sensitivity Analysis

The classification accuracy of remotely sensed data and its sensitivity to classification algorithms have a critical importance for the geospatial community, as classified images provide the base layers for many applications and models. Support Vector Machines (SVMs), a non-parametric statistical learning method that has recently been used in numerous applications in image processing. The SVMs need user-defined parameters and each parameter has different impact on kernels hence the classification accuracy of SVMs is based upon the choice of the parameters and kernels. The objective of this study is to investigate the sensitivity of SVM architecture including internal parameters and kernel types on landuse classification accuracy of RapidEye imagery for the study area in Turkey. Four types of kernels (linear, polynomial, radial basis function, and sigmoid) were used for the SVM classification. A total of 63 different models were developed and implemented for sensitivity analysis of SVM architecture. The traditional Maximum Likelihood Classification (MLC) method was also performed for comparison. The classification accuracies of the best model for each kernel type and MLC are 85.63%, 83.94%, 83.94%, 83.82% and 81.64% for polynomial, linear, radial basis function, sigmoid kernels and MLC, respectively. The results suggest that the choice of model parameters and kernel types play an important role on SVMs classification accuracy. Best model of polynomial kernel outperformed all SVMs models and gave the highest classification accuracy of 85.63% with RapidEye imagery.

Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control

In recent years the number of active controllable joints in electrically powered hand-prostheses has increased significantly. However, the control strategies for these devices in current clinical use are inadequate as they require separate and sequential control of each degree-of-freedom (DoF). In this study we systematically compare linear and nonlinear regression techniques for an independent, simultaneous and proportional myoelectric control of wrist movements with two DoF. These techniques include linear regression, mixture of linear experts (ME), multilayer-perceptron, and kernel ridge regression (KRR). They are investigated offline with electro-myographic signals acquired from ten able-bodied subjects and one person with congenital upper limb deficiency. The control accuracy is reported as a function of the number of electrodes and the amount and diversity of training data providing guidance for the requirements in clinical practice. The results showed that KRR, a nonparametric statistical learning method, outperformed the other methods. However, simple transformations in the feature space could linearize the problem, so that linear models could achieve similar performance as KRR at much lower computational costs. Especially ME, a physiologically inspired extension of linear regression represents a promising candidate for the next generation of prosthetic devices.

Bandwidth selection in kernel empirical risk minimization via the gradient

In this paper, we deal with the data-driven selection of multidimensional and possibly anisotropic bandwidths in the general framework of kernel empirical risk minimization. We propose a universal selection rule, which leads to optimal adaptive results in a large variety of statistical models such as nonparametric robust regression and statistical learning with errors in variables. These results are stated in the context of smooth loss functions, where the gradient of the risk appears as a good criterion to measure the performance of our estimators. The selection rule consists of a comparison of gradient empirical risks. It can be viewed as a nontrivial improvement of the so-called Goldenshluger-Lepski method to nonlinear estimators. Furthermore, one main advantage of our selection rule is the nondependency on the Hessian matrix of the risk, usually involved in standard adaptive procedures.

The use of classification trees for bioinformatics

Classification trees are non-parametric statistical learning methods that incorporate feature selection and interactions, possess intuitive interpretability, are efficient, and have high prediction accuracy when used in ensembles. This paper provides a brief introduction to the classification tree-based methods, a review of the recent developments, and a survey of the applications in bioinformatics and statistical genetics.

INDUCTION OF CLASSIFIERS THROUGH NON-PARAMETRIC METHODS FOR APPROXIMATE CLASSIFICATION AND RETRIEVAL WITH ONTOLOGIES

This work concerns non-parametric approaches for statistical learning applied to the standard knowledge representation languages adopted in the Semantic Web context. We present methods based on epistemic inference that are able to elicit and exploit the semantic similarity of individuals in OWL knowledge bases. Specifically, a totally semantic and language-independent semi-distance function is introduced, whence also an epistemic kernel function for Semantic Web representations is derived. Both the measure and the kernel function are embedded in non-parametric statistical learning algorithms customized for coping with Semantic Web representations. Particularly, the measure is embedded in a k-Nearest Neighbor algorithm and the kernel function is embedded in a Support Vector Machine. The implemented algorithms are used to perform inductive concept retrieval and query answering. An experimentation on real ontologies proves that the methods can be effectively employed for performing the target tasks, and moreover that it is possible to induce new assertions that are not logically derivable.

Biomimetic gaze stabilization based on feedback-error-learning with nonparametric regression networks

Oculomotor control in a humanoid robot faces similar problems as biological oculomotor systems, i.e. the stabilization of gaze in face of unknown perturbations of the body, selective attention, stereo vision, and dealing with large information processing delays. Given the nonlinearities of the geometry of binocular vision as well as the possible nonlinearities of the oculomotor plant, it is desirable to accomplish accurate control of these behaviors through learning approaches. This paper develops a learning control system for the phylogenetically oldest behaviors of oculomotor control, the stabilization reflexes of gaze. In a step-wise procedure, we demonstrate how control theoretic reasonable choices of control components result in an oculomotor control system that resembles the known functional anatomy of the primate oculomotor system. The core of the learning system is derived from the biologically inspired principle of feedback-error learning combined with a state-of-the-art non-parametric statistical learning network. With this circuitry, we demonstrate that our humanoid robot is able to acquire high performance visual stabilization reflexes after about 40 s of learning despite significant nonlinearities and processing delays in the system.