Class Of Support Vector Machines Research Articles

The Bernstein Mechanism: Function Release under Differential Privacy

We address the problem of general function release under differential privacy, by developing a functional mechanism that applies under the weak assumptions of oracle access to target function evaluation and sensitivity. These conditions permit treatment of functions described explicitly or implicitly as algorithmic black boxes. We achieve this result by leveraging the iterated Bernstein operator for polynomial approximation of the target function, and polynomial coefficient perturbation. Under weak regularity conditions, we establish fast rates on utility measured by high-probability uniform approximation. We provide a lower bound on the utility achievable for any functional mechanism that is epsilon-differentially private. The generality of our mechanism is demonstrated by the analysis of a number of example learners, including naive Bayes, non-parametric estimators and regularized empirical risk minimization. Competitive rates are demonstrated for kernel density estimation; and epsilon-differential privacy is achieved for a broader class of support vector machines than known previously.

A locality correlation preserving support vector machine

This paper proposes a locality correlation preserving based support vector machine (LCPSVM) by combining the idea of margin maximization between classes and local correlation preservation of class data. It is a Support Vector Machine (SVM) like algorithm, which explicitly considers the locality correlation within each class in the margin and the penalty term of the optimization function. Canonical correlation analysis (CCA) is used to reveal the hidden correlations between two datasets, and a variant of correlation analysis model which implements locality preserving has been proposed by integrating local information into the objective function of CCA. Inspired by the idea used in canonical correlation analysis, we propose a locality correlation preserving within-class scatter matrix to replace the within-class scatter matrix in minimum class variance support machine (MCVSVM). This substitution has the property of keeping the locality correlation of data, and inherits the properties of SVM and other similar modified class of support vector machines. LCPSVM is discussed under linearly separable, small sample size and nonlinearly separable conditions, and experimental results on benchmark datasets demonstrate its effectiveness.

On the stability and bias–variance analysis of sparse SVMs

Stability and bias–variance analysis are two powerful tools to better understand learning algorithms. We use these tools to analyze a class of support vector machines (SVMs) that try to reduce classifier complexity. The motivation for doing this is to compare the original and modified SVMs on two behavioral dimensions (a) stability and (b) learning behavior. Our preliminary experimental results show that (i) the class of algorithms which reduce classifier complexity by reducing the number of support vectors (SVs) are potentially unstable and (ii) the learning behavior is quite similar to the original SVMs.

Minimum Class Variance Support Vector Machines

In this paper, a modified class of support vector machines (SVMs) inspired from the optimization of Fisher's discriminant ratio is presented, the so-called minimum class variance SVMs (MCVSVMs). The MCVSVMs optimization problem is solved in cases in which the training set contains less samples that the dimensionality of the training vectors using dimensionality reduction through principal component analysis (PCA). Afterward, the MCVSVMs are extended in order to find nonlinear decision surfaces by solving the optimization problem in arbitrary Hilbert spaces defined by Mercer's kernels. In that case, it is shown that, under kernel PCA, the nonlinear optimization problem is transformed into an equivalent linear MCVSVMs problem. The effectiveness of the proposed approach is demonstrated by comparing it with the standard SVMs and other classifiers, like kernel Fisher discriminant analysis in facial image characterization problems like gender determination, eyeglass, and neutral facial expression detection.

Phase Space Prediction of Chaotic Time Series with Nu-Support VectorMachine Regression

A new class of support vector machine, nu-support vector machine,is discussed which can handle both classification and regression.We focus on nu-support vector machine regression and useit for phase space prediction of chaotic time series.The effectiveness of the method is demonstrated by applyingit to the Hénon map. This study also compares nu-supportvector machine with back propagation (BP) networksin order to better evaluate the performance of the proposed methods.The experimental results show that the nu-support vector machineregression obtains lower root mean squared error thanthe BP networks and provides an accurate chaotic timeseries prediction. These results can be attributableto the fact that nu-support vector machine implementsthe structural risk minimization principle and thisleads to better generalization than the BP networks.