Kernel Rule Research Articles

On the consistency of a new kernel rule for spatially dependent data

We consider a new kernel rule of classification for spatially dependent data. This nonparametric rule allows for the classification of missing observations. We investigate the consistency of this classification rule and we propose a method for bandwidth selection.

Pointwise adaptive estimation of the marginal density of a weakly dependent process

This paper is devoted to the estimation of the common marginal density function of weakly dependent processes. The accuracy of estimation is measured using pointwise risks. We propose a data-driven procedure using kernel rules. The bandwidth is selected using the approach of Goldenshluger and Lepski and we prove that the resulting estimator satisfies an oracle type inequality. The procedure is also proved to be adaptive (in a minimax framework) over a scale of Hölder balls for several types of dependence: strong mixing processes, λ-dependent processes or i.i.d. sequences can be considered using a single procedure of estimation. Some simulations illustrate the performance of the proposed method.

Solution Path for Pin-SVM Classifiers With Positive and Negative $\tau $ Values.

Applying the pinball loss in a support vector machine (SVM) classifier results in pin-SVM. The pinball loss is characterized by a parameter τ . Its value is related to the quantile level and different τ values are suitable for different problems. In this paper, we establish an algorithm to find the entire solution path for pin-SVM with different τ values. This algorithm is based on the fact that the optimal solution to pin-SVM is continuous and piecewise linear with respect to τ . We also show that the nonnegativity constraint on τ is not necessary, i.e., τ can be extended to negative values. First, in some applications, a negative τ leads to better accuracy. Second, τ = -1 corresponds to a simple solution that links SVM and the classical kernel rule. The solution for τ = -1 can be obtained directly and then be used as a starting point of the solution path. The proposed method efficiently traverses τ values through the solution path, and then achieves good performance by a suitable τ . In particular, τ = 0 corresponds to C-SVM, meaning that the traversal algorithm can output a result at least as good as C-SVM with respect to validation error.

Adaptive pointwise estimation of conditional density function

In this paper we consider the problem of estimating $f$, the conditional density of $Y$ given $X$, by using an independent sample distributed as $(X,Y)$ in the multivariate setting. We consider the estimation of $f(x,.)$ where $x$ is a fixed point. We define two different procedures of estimation, the first one using kernel rules, the second one inspired from projection methods. Both adapted estimators are tuned by using the Goldenshluger and Lepski methodology. After deriving lower bounds, we show that these procedures satisfy oracle inequalities and are optimal from the minimax point of view on anisotropic Holder balls. Furthermore, our results allow us to measure precisely the influence of $\mathrm{f}_X(x)$ on rates of convergence, where $\mathrm{f}_X$ is the density of $X$. Finally, some simulations illustrate the good behavior of our tuned estimates in practice.

Maxiset in sup-norm for kernel estimators

In the Gaussian white noise model, we study the estimation of an unknown multidimensional function f in the uniform norm by using kernel methods. We determine the sets of functions that are well estimated at the rates (log n/n) β/(2β+d) and n −β/(2β+d) by kernel estimators. These sets are called maxisets. Then, we characterize the maxisets associated to kernel estimators and to the Lepski procedure for the rate of convergence (log n/n) β/(2β+d) in terms of Besov and Hölder spaces of regularity β. Using maxiset results, optimal choices for the bandwidth parameter of kernel rules are derived. Performances of these rules are studied from the numerical point of view.

A kernel-based classifier on a Riemannian manifold

Abstract Let X be a random variable taking values in a compact Riemannian manifold without boundary, and let Y be a discrete random variable valued in {0;1} which represents a classification label. We introduce a kernel rule for classification on the manifold based on n independent copies of (X,Y). Under mild assumptions on the bandwidth sequence, it is shown that this kernel rule is consistent in the sense that its probability of error converges to the Bayes risk with probability one.

Classification Using Belief Functions: Relationship Between Case-Based and Model-Based Approaches

The transferable belief model (TBM) is a model to represent quantified uncertainties based on belief functions, unrelated to any underlying probability model. In this framework, two main approaches to pattern classification have been developed: the TBM model-based classifier, relying on the general Bayesian theorem (GBT), and the TBM case-based classifier, built on the concept of similarity of a pattern to be classified with training patterns. Until now, these two methods seemed unrelated, and their connection with standard classification methods was unclear. This paper shows that both methods actually proceed from the same underlying principle, i.e., the GBT, and that they essentially differ by the nature of the assumed available information. This paper also shows that both methods collapse to a kernel rule in the case of precise and categorical learning data and for certain initial assumptions, and a simple relationship between basic belief assignments produced by the two methods is exhibited in a special case. These results shed new light on the issues of classification and supervised learning in the TBM. They also suggest new research directions and may help users in selecting the most appropriate method for each particular application, depending on the nature of the information at hand.

On the Kernel Rule for Function Classification

Let X be a random variable taking values in a function space $$\mathcal{F}$$ , and let Y be a discrete random label with values 0 and 1. We investigate asymptotic properties of the moving window classification rule based on independent copies of the pair (X,Y). Contrary to the finite dimensional case, it is shown that the moving window classifier is not universally consistent in the sense that its probability of error may not converge to the Bayes risk for some distributions of (X,Y). Sufficient conditions both on the space $$\mathcal{F}$$ and the distribution of X are then given to ensure consistency.

Road sign classification using Laplace kernel classifier

Driver support systems (DSS) of intelligent vehicles will predict potentially dangerous situations in heavy traffic, help with navigation and vehicle guidance and interact with a human driver. Important information necessary for traffic situation understanding is presented by road signs. A new kernel rule has been developed for road sign classification using the Laplace probability density. Smoothing parameters of the Laplace kernel are optimized by the pseudo-likelihood cross-validation method. To maximize the pseudo-likelihood function, an Expectation–Maximization algorithm is used. The algorithm has been tested on a dataset with more than 4900 noisy images. A comparison to other classification methods is also given.

Predictive dynamic load balancing of parallel and distributed rule and query processing

Expert Databases are environments that support the processing of rule programs against a disk resident database. They occupy a position intermediate between active and deductive databases, with respect to the level of abstraction of the underlying rule language. The operational semantics of the rule language influences the problem solving strategy, while the architecture of the processing environment determines efficiency and scalability. In this paper, we present elements of the PARADISER architecture and its kernel rule language, PARULEL. The PARADISER environment provides support for parallel and distributed evaluation of rule programs, as well as static and dynamic load balancing protocols that predictively balance a computation at runtime. This combination of features results in a scalable database rule and complex query processing architecture. We validate our claims by analyzing the performance of the system for two realistic test cases. In particular, we show how the performance of a parallel implementation of transitive closure is significantly improved by predictive dynamic load balancing.

Remarks on Non-Parametric Estimates for Density Functions and Regression Curves

Remarks on Non-Parametric Estimates for Density Functions and Regression Curves