Multidimensional Support Vector Regression Research Articles

Research on Location Algorithm Based on Beacon Filtering Combining DV-Hop and Multidimensional Support Vector Regression.

The DV-Hop algorithm is widely used because of its simplicity and low cost, but it has the disadvantage of a large positioning error. In recent years, although some improvement measures have been proposed, such as hop correction, distance-weighted correction, and improved coordinate solution, there is room for improvement in location accuracy, and the accuracy is affected in anisotropic networks. A location algorithm based on beacon filtering combining DV-Hop and multidimensional support vector regression (MSVR) is proposed in this paper. In the process of estimating the coordinates of unknown nodes, received signal strength indication (RSSI), MSVR, and weighted least squares method are combined. In addition, the verification error of beacon nodes is proposed, which can select the beacon nodes with smaller errors to reduce the location error. Simulation results show that in different distributions, the location accuracy of the proposed algorithm is at least 34% higher than that of the classical DV-Hop algorithm and at least 28% higher than that of the localization based on multidimensional support vector regression (LMSVR) algorithm. The proposed algorithm has the potential of application in small-scale anisotropic networks.

Novel Prediction Model for Steel Mechanical Properties with MSVR Based on MIC and Complex Network Clustering

Traditional mechanical properties prediction models are mostly based on experience and mechanism, which neglect the linear and nonlinear relationships between process parameters. Aiming at the high-dimensional data collected in the complex industrial process of steel production, a new prediction model is proposed. The multidimensional support vector regression (MSVR)-based model is combined with the feature selection method, which involves maximum information coefficient (MIC) correlation characterization and complex network clustering. Firstly, MIC is used to measure the correlation between process parameters and mechanical properties, based on which a complex network is constructed and hierarchical clustering is performed. Secondly, we evaluate all parameters and select a representative one for each partition as the input of the subsequent model based on the centrality and influence indicators. Finally, an actual steel production case is used to train the MSVR prediction model. The prediction results show that our proposed framework can capture effective features from the full parameters in terms of higher prediction accuracy and is less time-consuming compared with the Pearson-based subset, full-parameter subset, and empirical subset input. The feature selection method based on MIC can dig out some nonlinear relationships which cannot be found by Pearson coefficient.

Hybrid machine learning algorithm and statistical time series model for network-wide traffic forecast

We propose a novel approach for network-wide traffic state prediction where the statistical time series model ARIMA is used to postprocess the residuals out of the fundamental machine learning algorithm MLP. This approach is named as NN-ARIMA. Neural Network MLP is employed to capture network-scale co-movement pattern of all traffic flows, and ARIMA is used to further extract location-specific traffic features in the residual time series out of Neural Network. The experiment results show that the postprocessing the residuals of Neural Network by the ARIMA analysis helps to significantly improve accuracy of traffic state prediction by 8.9–13.4% in term of mean squared error reduction. In order to verify the efficiency of the ARIMA analysis in the postprocessing, Multidimensional Support Vector Regression (MSVR) model is also employed to replace the role of Neural Network in the comparative experiment. Two streams of comparisons, (1) NN vs. NN-ARIMA and (2) MSVR vs. MSVR-ARIMA, are performed and show consistent results. The proposed approach not only can capture network-wide co-movement pattern of traffic flows, but also seize location-specific traffic characteristics as well as sharp nonlinearity of macroscopic traffic variables. The case study indicates that the accuracy of prediction can be significantly improved when both network-scale traffic features and location-specific characteristics are taken into account.

A Novel Group Recommendation Mechanism From the Perspective of Preference Distribution

As people have become more and more connected, there are certain scenarios where items need to be recommended to groups of users rather than individual user, which motivate studies on group recommender systems (GRSs). However, developing GRSs is not an easy task, because a group consists of multiple members with heterogeneous preferences. How to make a trade-off among their preferences remains challenging. Existing works almost aggregate members’ preferences into forms of single values as group profile. However, simple aggregations fail to well reflect comprehensive group profile when it comes to groups with highly conflicting preferences. In this paper, we propose Greption, a novel group recommendation mechanism from the perspective of preference distribution. First, based on preference distributions toward items in training set, a multi-dimensional support vector regression model is established to predict preference distributions toward candidate items. Then, through a modified VIKOR method, we transform the process of selecting items for a group into a multi-criteria decision making process. Furthermore, the Greption is extended to be able to handle data sparsity. Specifically, we propose two heuristic schemes for this purpose. And we present a set of experiments to evaluate the efficiency of the Greption.

MSVR Based Range-Free Localization Technique for 3-D Sensor Networks

Location estimation is essential in any Wireless Sensor Network (WSN). A novel range-free localization approach has been proposed in this research that is based on multidimensional support vector regression (MSVR). To solve the regression problem, in this research a new training method for MSVR is proposed. The proposed localization approach is formulated in such a manner that the unknown sensor node position is calculated by using the position information of known actuator node by using proximity measurements. The simulations for the proposed schemes are done for both anisotropic and isotropic networks and also for both 2-D and 3-D environments. The simulated results showed the excellent performance by the proposed schemes in various scenarios as compared with the already existing schemes.

Reference tag supported RFID tracking using robust support vector regression and Kalman filter

Site operations usually contain potential safety issues and an effective monitoring strategy for operations is essential to predict and prevent risk. Regarding the status monitoring among material, equipment and personnel during site operations, much work is conducted on localization and tracking using Radio Frequency Identification (RFID) technology. However, existing RFID tracking methods suffer from low accuracy and instability, due to severe interference in industrial sites with many metal structures. To improve RFID tracking performance in industrial sites, a RFID tracking method that integrates Multidimensional Support Vector Regression (MSVR) and Kalman filter is developed in this paper. Extensive experiments have been conducted on a Liquefied Natural Gas (LNG) facility site with long range active RFID system to evaluate the performance of this approach. The results demonstrate the effectiveness and stability of the proposed approach with severe noise and outliers. It is feasible to adopt the proposed approach which satisfies intrinsically-safe regulations for monitoring operation status in current practice.

Multi-input-multi-output support vector machine based on principal curve

To solve the problem that the traditional Multi-Input-Multi-Output(MIMO) Support Vector Machine(SVM) generally ignore the dependency among all outputs,a new MIMO SVM algorithm based on principal curve was proposed in this paper.Following the assumption that the model parameters of all outputs locate on a manifold,this paper firstly constructed a manifold regularization based on the Multi-dimensional Support Vector Regression(M-SVR),where the regularizer was the squared distance from the output parameters to the principal curve through the middle of all parameters' set.Secondly,considering the non-convexity of this regularization,this paper introduced an alternative optimization method to calculate the model parameters and principal curve in turn until convergence.The experiments on simulated data and real-life dynamic load identification data were conducted,and the results show that the proposed algorithm performs better than M-SVR and SVM based separate modeling method in terms of prediction precision and numerical stability.

A fast and robust model selection algorithm for multi-input multi-output support vector machine

Multi-Input Multi-Output (MIMO) regression estimation problems widely exist in engineering fields. As an efficient approach for MIMO modeling, multi-dimensional support vector regression, named M-SVR, is generally capable of obtaining better predictions than many traditional methods. However, M-SVR is sensitive to the perturbation of hyper-parameters when facing small-scale sample problems, and most of currently used model selection methods for conventional SVR cannot be applied to M-SVR directly due to its special structure. In this paper, a fast and robust model selection algorithm for M-SVR is proposed. Firstly, a new training algorithm for M-SVR is proposed to reduce efficiently the numerical errors in training procedure. Based on this algorithm, a new leave-one-out (LOO) error estimate for M-SVR is derived through a virtual LOO cross-validation procedure. This LOO error estimate can be straightway calculated once a training process ended with less computational complexity than traditional LOO method. Furthermore, a robust implementation of this LOO estimate via Cholesky factorization is also proposed. Finally, the gradients of the LOO estimate are calculated, and the hyper-parameters with lowest LOO error can be found by means of gradient decent method. Experiments on toy data and real-life dynamical load identification problems are both conducted, demonstrating comparable results of the proposed algorithm in terms of generalization performance, numerical stability and computational cost.

Novel range-free localization based on multidimensional support vector regression trained in the primal space.

A novel range-free localization algorithm based on the multidimensional support vector regression (MSVR) is proposed in this paper. The range-free localization problem is formulated as a multidimensional regression problem, and a new MSVR training method is proposed to solve the regression problem. Unlike standard support vector regression, the proposed MSVR allows multiple outputs and localizes the sensors without resorting to multilateration. The training of the MSVR is formulated directly in primal space and it can be solved in two ways. First, it is formulated as a second-order cone programming and trained by convex optimization. Second, its own training method is developed based on the Newton-Raphson method. A simulation is conducted for both isotropic and anisotropic networks, and the proposed method exhibits excellent and robust performance in both isotropic and anisotropic networks.

A new kernelized approach to wireless sensor network localization

In this paper, a new approach to range-free localization in Wireless Sensor Networks (WSNs) is proposed using nonlinear mapping, and the kernel function is introduced. The localization problem in the WSN is formulated as a kernelized regression problem, which is solved by support vector regression (SVR) and multi-dimensional support vector regression (MSVR). The proposed methods are simple and efficient in that no additional hardware is required for the measurements, and only proximity information and position information of the anchor nodes are used for the localization. The proposed methods are composed of three steps: the measurement step, kernelized regression step, and localization step. In the measurement step, the proximity information of the given network is measured. In the regression step, the relationships among the geographical distances and the proximity among sensor nodes is built using kernelized regression. In the localization step, each sensor node finds its own position in a distributed manner using a kernelized regressor. The simulation result demonstrates that the proposed methods exhibit excellent and robust location estimation performance.

Leave-one-out cross-validation-based model selection for multi-input multi-output support vector machine

As an effective approach for multi-input multi-output regression estimation problems, a multi-dimensional support vector regression (SVR), named M-SVR, is generally capable of obtaining better predictions than applying a conventional support vector machine (SVM) independently for each output dimension. However, although there are many generalization error bounds for conventional SVMs, all of them cannot be directly applied to M-SVR. In this paper, a new leave-one-out (LOO) error estimate for M-SVR is derived firstly through a virtual LOO cross-validation procedure. This LOO error estimate can be straightway calculated once a training process ended with less computational complexity than traditional LOO method. Based on this LOO estimate, a new model selection methods for M-SVR based on multi-objective optimization strategy is further proposed in this paper. Experiments on toy noisy function regression and practical engineering data set, that is, dynamic load identification on cylinder vibration system, are both conducted, demonstrating comparable results of the proposed method in terms of generalization performance and computational cost.

A Division Algebraic Framework for Multidimensional Support Vector Regression

In this paper, division algebras are proposed as an elegant basis upon which to extend support vector regression (SVR) to multidimensional targets. Using this framework, a multitarget SVR called epsilon(Z)-SVR is proposed based on an epsilon-insensitive loss function that is independent of the coordinate system or basis used. This is developed to dual form in a manner that is analogous to the standard epsilon-SVR. The epsilon(H)-SVR is compared and contrasted with the least-square SVR (LS-SVR), the Clifford SVR (C-SVR), and the multidimensional SVR (M-SVR). Three practical applications are considered: namely, 1) approximation of a complex-valued function; 2) chaotic time-series prediction in 3-D; and 3) communication channel equalization. Results show that the epsilon(H)-SVR performs significantly better than the C-SVR, the LS-SVR, and the M-SVR in terms of mean-squared error, outlier sensitivity, and support vector sparsity.