Non-convex Loss Function Research Articles

A Robust Boosting Algorithm for Chemical Modeling

Bagging and boosting have become increasingly important ensemble methods for combining models in the data mining and machine learning literature. We review the basic ideas of these methods, propose a new robust boosting algorithm based on a non-convex loss function and compare the performance of these methods to both simulated and real data sets with and without contamination.

Estimating the ratio of two scale parameters: a simple approach

We describe a simple approach for estimating the ratio ρ = σ2/σ1 of the scale parameters of two populations from a decision theoretic point of view. We show that if the loss function satisfies a certain condition, then the estimation of ρ reduces to separately estimating σ2 and 1/σ1. This implies that the standard estimator of ρ can be improved by just employing an improved estimator of σ2 or 1/σ1. Moreover, in the case where the loss function is convex in some function of its argument, we prove that such improved estimators of ρ are further dominated by corresponding ones that use all the available data. Using this result, we construct new classes of double-adjustment improved estimators for several well-known convex as well as non-convex loss functions. In particular, Strawderman-type estimators of ρ in general models are given whereas Shinozaki-type estimators of the ratio of two normal variances are briefly treated.

Robust truncated support vector regression

In this paper, we utilize two ε-insensitive loss functions to construct a non-convex loss function. Based on this non-convex loss function, a robust truncated support vector regression (TSVR) is proposed. In order to solve the TSVR, the concave–convex procedure is used to circumvent this problem though transforming the non-convex problem to a sequence of convex ones. The TSVR owns better robustness to outliers than the classical support vector regression, which makes the TSVR gain advantages in the generalization ability and the number of support vector. Finally, the experiments on the synthetic and real-world benchmark data sets further confirm the effectiveness of our proposed TSVR.

Support vector machine classification with indefinite kernels

In this paper, we propose a method for support vector machine classification using indefinite kernels. Instead of directly minimizing or stabilizing a nonconvex loss function, our method simultaneously finds the support vectors and a proxy kernel matrix used in computing the loss. This can be interpreted as a robust classification problem where the indefinite kernel matrix is treated as a noisy observation of the true positive semidefinite kernel. Our formulation keeps the problem convex and relatively large problems can be solved efficiently using the analytic center cutting plane method. We compare the performance of our technique with other methods on several data sets.

Robust support vector regression in the primal

The classical support vector regressions (SVRs) are constructed based on convex loss functions. Since non-convex loss functions to a certain extent own superiority to convex ones in generalization performance and robustness, we propose a non-convex loss function for SVR, and then the concave-convex procedure is utilized to transform the non-convex optimization to convex one. In the following, a Newton-type optimization algorithm is developed to solve the proposed robust SVR in the primal, which can not only retain the sparseness of SVR but also oppress outliers in the training examples. The effectiveness, namely better generalization, is validated through experiments on synthetic and real-world benchmark data sets.

Nonconvexity in noncooperative game theory

Recent modeling efforts which attempted to apply the classical Nash equilibrium theory were found unsatisfactory because one or more players in the game had nonconvex loss functions (or nonconcave utility functions). In order to handle these difficult and important models, a generalization of the equilibrium is proposed which is denoted the pre-equilibrium set. This set contains all solutions to the first-order necessary optimality conditions for each player's utility function with respect to every opponent's necessary optimality conditions solutions. It contains all the Nash equilibria, as well as other points which may be of interest in their own right.