Support Vector Regression Problem Research Articles

IMPLEMENTASI GRIDSEARCHCV PADA SUPPORT VECTOR REGRESSION (SVR) UNTUK PERAMALAN HARGA SAHAM

Stock is a sign of the capital participation of a person or authority in a company (PT). PT Anabatic Technologies Tbk (ATIC) is one of the service providers and IT consultants that is included in the technology sector, which is a new sector in the IDX-IC classification. ATIC stock trading was temporarily suspended due to a significant increase in cumulative prices. This indicates that stock prices tend to be volatile and non-linear. The Support Vector Regression (SVR) method can be used to predict stock prices. SVR is able to solve non-linear data problems by using kernel functions so it can overcome overfitting problems and will give good performance. The SVR problem is difficult to determine the optimal hyperparameters, so this research uses grid search cross validation (GridSearchCV). In this research, ATIC’s daily closing price data was used with 1007 training data and 252 testing data. The results show that the best model is SVR with a linear kernel and the hyperparameters used are Cost and epsilon . The linear kernel SVR model produces a MSE of 0,001237173; SMAPE of 0,1167301; and = 0,9206643

Quantum Support Vector Regression for Disability Insurance

We propose a hybrid classical-quantum approach for modeling transition probabilities in health and disability insurance. The modeling of logistic disability inception probabilities is formulated as a support vector regression problem. Using a quantum feature map, the data are mapped to quantum states belonging to a quantum feature space, where the associated kernel is determined by the inner product between the quantum states. This quantum kernel can be efficiently estimated on a quantum computer. We conduct experiments on the IBM Yorktown quantum computer, fitting the model to disability inception data from a Swedish insurance company.

A new method of linear support vector regression with interval data

‎In this paper‎, ‎the linear support vector regression approach is proposed for solving the regression problem with interval data‎, ‎which is called interval support vector regression(ISVR)‎. ‎The ISVR approach is equivalent to solving a linear constrained quadratic programming problem (QPP) with an interval cost coefficient in which the value of the objective function is in an interval‎. ‎Instead of solving an interval QPP‎, ‎we solve two QPPs and prove that the cost values of these two problems are the lower bound and the upper bound of the target value of the interval QPP‎. ‎We show these two mentioned QPPs are equivalent to two support vector regression problems which the first problem applies the lower bound of data and the second problem considers the upper bound of the data‎. ‎to obtain the regression function‎. ‎Some experiments are made to demonstrate the performance of our method compared with the known algorithms on several artificial‎, ‎benchmark and real practical datasets‎.‎‎‎

Adaptive Bayesian support vector regression model for structural reliability analysis

In this paper, Bayesian support vector regression (SVR) model is developed for structural reliability analysis adaptively. Two SVR models, namely, least-square SVR and ε-SVR, are constructed under the Bayesian inference framework with a square loss function and a ε-insensitive square one respectively. In this framework, a Gaussian process prior is assigned to the regression function, and maximum posterior estimate results in a SVR problem. The proposed Bayesian SVR models provide point-wise probabilistic prediction while keeps the structural risk minimization principle, and it allows us to determine the optimal hyper-parameters by maximizing Bayesian model evidence. Two active learning algorithms are presented based on the Bayesian SVR models to estimate large and small failure probability of complex structure with limited model evaluations respectively. Four benchmark examples are employed to validate the performance of the presented method.

Active learning Bayesian support vector regression model for global approximation

Machine learning techniques have received much attention in many areas for regression and classification tasks. In this paper, two new support vector regression (SVR) models, namely, least-square SVR and ε-SVR, are developed under the Bayesian inference framework with a square loss function and a ε-insensitive squared one respectively. In this framework, a Gaussian process prior is assigned to the regression function, and maximum posterior estimate of this function results in a support vector regression problem. The proposed method provides point-wise probabilistic prediction while keeps the structural risk minimization principle, and it allows us to determine the optimal hyper-parameters by maximizing Bayesian model evidence. Based on the Bayesian SVR model, an active learning algorithm is developed, and new training points are selected adaptively based on a learning function to update the SVR model progressively. Numerical results reveal that the developed two Bayesian SVR models are very promising for constructing accurate regression model for problems with diverse characteristics, especially for medium and high dimensional problems.

A cost-effective approach to portfolio construction with range-based risk measures

In this paper, we introduce a new class of risk measures and the corresponding risk minimizing portfolio optimization problem. Instead of measuring the expected deviation of a daily return from a single target value, we propose to measure its deviation from a range of values centered on the single target value. By relaxing the definition of deviation, the proposed risk measure is robust to the variation of data input and thus the resulting risk-minimizing portfolio has a lower turnover rate and is resilient to outliers. To construct a practical portfolio, we propose to impose an -norm constraint on the portfolio weights to stabilize the portfolio's out-of-sample performance. We show that for some cases of our proposed range-based risk measures, the corresponding portfolio optimization can be recast as a support vector regression problem. This allows us to tap into the machine learning literature on support vector regression and effectively solve the portfolio optimization problem even in high dimensions. Moreover, we present theoretical results on the robustness of our range-based risk minimizing portfolios. Simulation and empirical studies are conducted to examine the out-of-sample performance of the proposed portfolios.

A neurodynamic approach to nonsmooth constrained pseudoconvex optimization problem

This paper presents a new neurodynamic approach for solving the constrained pseudoconvex optimization problem based on more general assumptions. The proposed neural network is equipped with a hard comparator function and a piecewise linear function, which make the state solution not only stay in the feasible region, but also converge to an optimal solution of the constrained pseudoconvex optimization problem. Compared with other related existing conclusions, the neurodynamic approach here enjoys global convergence and lower dimension of the solution space. Moreover, the neurodynamic approach does not depend on some additional assumptions, such as the feasible region is bounded, the objective function is lower bounded over the feasible region or the objective function is coercive. Finally, both numerical illustrations and simulation results in support vector regression problem show the well performance and the viability of the proposed neurodynamic approach.

A Cost-effective Approach to Portfolio Construction with Range-based Risk Measures

In this paper, we introduce a new class of risk measures and the corresponding risk minimizing portfolio optimization problem. Instead of measuring the expected deviation of a daily return from a single target value, we propose to measure its deviation from a range of values centered on the single target value. By relaxing the definition of deviation, the proposed risk measure is robust to the variation of data input and thus the resulting risk-minimizing portfolio has a lower turnover rate and is resilient to outliers. To construct a practical portfolio, we propose to impose an l2-norm constraint on the portfolio weights to stabilize the portfolio's out-of-sample performance. We show that for some cases of our proposed range-based risk measures, the corresponding portfolio optimization can be recast as a support vector regression problem. This allows us to tap into the machine learning literature on support vector regression and effectively solve the portfolio optimization problem even in high dimensions. Moreover, we present theoretical results on the robustness of our range-based risk minimizing portfolios. Simulation and empirical studies are conducted to examine the out-of-sample performance of the proposed portfolios.

Multiple Radar Subbands Fusion Algorithm Based on Support Vector Regression in Complex Noise Environment

Real noise environment may not be Gaussian due to the existence of man-made interferences, natural bursts, and so on. In this paper, a new multiple radar subbands fusion algorithm in complex noise environment is proposed. The considered complex noise includes impulsive noise and the mixture of impulsive and Gaussian noises. The proposed subband fusion algorithm consists of the following steps. First, the incoherent factors are estimated and the multiple subbands are compensated to be mutually coherent. Then, coherent subbands are fit by the geometrical theory of diffraction (GTD) model, where model parameters are estimated in the frame of sparse reconstruction. Different probability distribution functions are used to describe different noise environments, and the maximum a posteriori probability (MAP) estimate of scattering model parameters is achieved. It is shown that the MAP estimate can be obtained by iteratively solving a support vector regression problem. Finally, the fused full band is obtained by extrapolating the measured subband data based on the estimated GTD model. Validity and performance of the proposed algorithm are investigated by analytical data, simulated data, and static-range data.

An application of a practical neural network model for solving support vector regression problems

An application of a practical neural network model for solving support vector regression problems

On Weighted Support Vector Regression

We propose a new type of weighted support vector regression (SVR), motivated by modeling local dependencies in time and space in prediction of house prices. The classic weights of the weighted SVR are added to the slack variables in the objective function (OF‐weights). This procedure directly shrinks the coefficient of each observation in the estimated functions; thus, it is widely used for minimizing influence of outliers. We propose to additionally add weights to the slack variables in the constraints (CF‐weights) and call the combination of weights the doubly weighted SVR. We illustrate the differences and similarities of the two types of weights by demonstrating the connection between the Least Absolute Shrinkage and Selection Operator (LASSO) and the SVR. We show that an SVR problem can be transformed to a LASSO problem plus a linear constraint and a box constraint. We demonstrate the capabilities of the doubly weighted approach through an example of prediction of house prices. The weight functions in the house pricing model depend on the geographical distance to the house of interest and the difference in time of sale (CF‐weights) as well as the differences lying in variables (OF‐weights), such as house size and number of floors. The results illustrate that the combination of the two types of weights describes the relative importance of observations very well and lowers the influence of possible outliers. Therefore, it enables the SVR models to have good performance. Copyright © 2014 John Wiley & Sons, Ltd.

An algorithm for training a large scale support vector machine for regression based on linear programming and decomposition methods

This paper presents a method to train a Support Vector Regression (SVR) model for the large-scale case where the number of training samples supersedes the computational resources. The proposed scheme consists of posing the SVR problem entirely as a Linear Programming (LP) problem and on the development of a sequential optimization method based on variables decomposition, constraints decomposition, and the use of primal–dual interior point methods. Experimental results demonstrate that the proposed approach has comparable performance with other SV-based classifiers. Particularly, experiments demonstrate that as the problem size increases, the sparser the solution becomes, and more computational efficiency can be gained in comparison with other methods. This demonstrates that the proposed learning scheme and the LP-SVR model are robust and efficient when compared with other methodologies for large-scale problems.

Model selection for primal SVM

This paper introduces two types of nonsmooth optimization methods for selecting model hyperparameters in primal SVM models based on cross-validation. Unlike common grid search approaches for model selection, these approaches are scalable both in the number of hyperparameters and number of data points. Taking inspiration from linear-time primal SVM algorithms, scalability in model selection is achieved by directly working with the primal variables without introducing any dual variables. The proposed implicit primal gradient descent (ImpGrad) method can utilize existing SVM solvers. Unlike prior methods for gradient descent in hyperparameters space, all work is done in the primal space so no inversion of the kernel matrix is required. The proposed explicit penalized bilevel programming (PBP) approach optimizes both the hyperparameters and parameters simultaneously. It solves the original cross-validation problem by solving a series of least squares regression problems with simple constraints in both the hyperparameter and parameter space. Computational results on least squares support vector regression problems with multiple hyperparameters establish that both the implicit and explicit methods perform quite well in terms of generalization and computational time. These methods are directly applicable to other learning tasks with differentiable loss functions and regularization functions. Both the implicit and explicit algorithms investigated represent powerful new approaches to solving large bilevel programs involving nonsmooth loss functions.

Optimization problems in statistical learning: Duality and optimality conditions

Regularization methods are techniques for learning functions from given data. We consider regularization problems the objective function of which consisting of a cost function and a regularization term with the aim of selecting a prediction function f with a finite representation f ( · ) = ∑ i = 1 n c i k ( · , X i ) which minimizes the error of prediction. Here the role of the regularizer is to avoid overfitting. In general these are convex optimization problems with not necessarily differentiable objective functions. Thus in order to provide optimality conditions for this class of problems one needs to appeal on some specific techniques from the convex analysis. In this paper we provide a general approach for deriving necessary and sufficient optimality conditions for the regularized problem via the so-called conjugate duality theory. Afterwards we employ the obtained results to the Support Vector Machines problem and Support Vector Regression problem formulated for different cost functions.

A novel force field parameter optimization method based on LSSVR for ECEPP

In this paper, we propose a novel force field parameter optimization method based on LSSVR and optimize the torsion energy parameters of ECEPP force field. In this method force field parameter optimization problem is turned into a support vector regression problem. Protein samples for regression model training are chosen from Protein Data Bank. The experiments show that the optimized force-field parameters make both α-helix and β-hairpin structures more consistent with the experimental implications than the original parameters.

Identification of state-dependent parameter models with support vector regression

A support vector regression approach is presented for the identification of state-dependent parameter ARX models, whose parameters are described as functions of past inputs and outputs. The problem of identifying the state-dependent parameters reduces to a standard support vector regression problem with a kernel function which is defined in terms of the kernels used to represent the individual parameters. Numerical examples show that the support vector method gives accurate parameter estimates for systems which have a state-dependent parameter representation.

Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR)

A parallel randomized support vector machine (PRSVM) and a parallel randomized support vector regression (PRSVR) algorithm based on a randomized sampling technique are proposed in this paper. The proposed PRSVM and PRSVR have four major advantages over previous methods. (1) We prove that the proposed algorithms achieve an average convergence rate that is so far the fastest bounded convergence rate, among all SVM decomposition training algorithms to the best of our knowledge. The fast average convergence bound is achieved by a unique priority based sampling mechanism. (2) Unlike previous work (Provably fast training algorithm for support vector machines, 2001) the proposed algorithms work for general linear-nonseparable SVM and general non-linear SVR problems. This improvement is achieved by modeling new LP-type problems based on Karush–Kuhn–Tucker optimality conditions. (3) The proposed algorithms are the first parallel version of randomized sampling algorithms for SVM and SVR. Both the analytical convergence bound and the numerical results in a real application show that the proposed algorithm has good scalability. (4) We present demonstrations of the algorithms based on both synthetic data and data obtained from a real word application. Performance comparisons with SVMlight show that the proposed algorithms may be efficiently implemented.

Image Superresolution Using Support Vector Regression

A thorough investigation of the application of support vector regression (SVR) to the superresolution problem is conducted through various frameworks. Prior to the study, the SVR problem is enhanced by finding the optimal kernel. This is done by formulating the kernel learning problem in SVR form as a convex optimization problem, specifically a semi-definite programming (SDP) problem. An additional constraint is added to reduce the SDP to a quadratically constrained quadratic programming (QCQP) problem. After this optimization, investigation of the relevancy of SVR to superresolution proceeds with the possibility of using a single and general support vector regression for all image content, and the results are impressive for small training sets. This idea is improved upon by observing structural properties in the discrete cosine transform (DCT) domain to aid in learning the regression. Further improvement involves a combination of classification and SVR-based techniques, extending works in resolution synthesis. This method, termed kernel resolution synthesis, uses specific regressors for isolated image content to describe the domain through a partitioned look of the vector space, thereby yielding good results.

Splines in Higher Order TV Regularization

Splines play an important role as solutions of various interpolation and approximation problems that minimize special functionals in some smoothness spaces. In this paper, we show in a strictly discrete setting that splines of degree m?1 solve also a minimization problem with quadratic data term and m-th order total variation (TV) regularization term. In contrast to problems with quadratic regularization terms involving m-th order derivatives, the spline knots are not known in advance but depend on the input data and the regularization parameter ?. More precisely, the spline knots are determined by the contact points of the m---th discrete antiderivative of the solution with the tube of width 2? around the m-th discrete antiderivative of the input data. We point out that the dual formulation of our minimization problem can be considered as support vector regression problem in the discrete counterpart of the Sobolev space W 2,0 m . From this point of view, the solution of our minimization problem has a sparse representation in terms of discrete fundamental splines.

Active Set Support Vector Regression

This paper presents active set support vector regression (ASVR), a new active set strategy to solve a straightforward reformulation of the standard support vector regression problem. This new algorithm is based on the successful ASVM algorithm for classification problems, and consists of solving a finite number of linear equations with a typically large dimensionality equal to the number of points to be approximated. However, by making use of the Sherman-Morrison-Woodbury formula, a much smaller matrix of the order of the original input space is inverted at each step. The algorithm requires no specialized quadratic or linear programming code, but merely a linear equation solver which is publicly available. ASVR is extremely fast, produces comparable generalization error to other popular algorithms, and is available on the web for download.