Multi-output Support Vector Regression Research Articles

Human knee joint walking pattern generation using computational intelligence techniques

Computational intelligence techniques (CITs) can be used to generate the human knee joint angle walking pattern in the sagittal plane, useful in medical rehabilitation as a specific reference of normal pattern depending on the subject’s age, mass, height and stride duration. In this paper, the knee joint angle reference curves in the sagittal plane were generated by using three different CITs: artificial neural network, extreme learning machine (ELM) and multi-output support vector regression. The gait pattern of a woman is different of the gait pattern of a man, and consequently, their knee joint angle curves are also different. Thus, it was necessary to train and test each of the three CITs for each gender. The data used by the CIT were obtained from volunteers with healthy gait and with different characteristics (gender, age, height and weight). The volunteers’ knee joint angle curves were collected by a system mainly constituted by a treadmill, two web cameras and passive marks positioned at volunteers’ joints. These gait analyses were made for five different walking speeds. It was observed that the best curves for each gender were generated using the ELM. Therefore, the ELM can be used to generate the normal knee joint angle curves expected for any person with specific characteristics (age, mass, height, stride duration), and physicians can use these specific normal curves for comparison purposes instead of using the standard knee joint angle curves of the literature which do not take into consideration the specific characteristics of the joint angle source.

Subpixel Mapping of Urban Areas Using EnMAP Data and Multioutput Support Vector Regression

Hyperspectral remote sensing data offer the opportunity to map urban characteristics in detail. Though, adequate algorithms need to cope with increasing data dimensionality, high redundancy between individual bands, and often spectrally complex urban landscapes. The study focuses on subpixel quantification of urban land cover compositions using simulated environmental mapping and analysis program (EnMAP) data acquired over the city of Berlin, utilizing both machine learning regression and classification algorithms, i.e., multioutput support vector regression (MSVR), standard support vector regression (SVR), import vector machine classifier (IVM), and support vector classifier (SVC). The experimental setup incorporates a spectral library and a reference land cover fraction map used for validation purposes. The library spectra were synthetically mixed to derive quantitative training data for the classes vegetation, impervious surface, soil, and water. MSVR and SVR models were trained directly using the synthetic mixtures. For IVM and SVC, a modified hyperparameter selection approach is conducted to improve the description of urban land cover fractions by means of probability outputs. Validation results demonstrate the high potential of the MSVR for subpixel mapping in the urban context. MSVR outperforms SVR in terms of both accuracy and computational time. IVM and SVC work similarly well, yet with lower accuracies of subpixel fraction estimates compared to both regression approaches.

Self-training for multi-target regression with tree ensembles

Semi-supervised learning (SSL) aims to use unlabeled data as an additional source of information in order to improve upon the performance of supervised learning methods. The availability of labeled data is often limited due to the expensive and/or tedious annotation process, while unlabeled data could be easily available in large amounts. This is particularly true for predictive modelling problems with a structured output space. In this study, we address the task of SSL for multi-target regression (MTR), where the output space consists of multiple numerical values. We extend the self-training approach to perform SSL for MTR by using a random forest of predictive clustering trees. In self-training, a model iteratively uses its own most reliable predictions, hence a good measure for the reliability of predictions is essential. Given that reliability estimates for MTR predictions have not yet been studied, we propose four such estimates, based on mechanisms provided within ensemble learning. In addition to these four scores, we use two benchmark scores (oracle and random) to empirically determine the performance limits of self-training. We also propose an approach to automatically select a threshold for the identification of the most reliable predictions to be used in the next iteration. An empirical evaluation on a large collection of datasets for MTR shows that self-training with any of the proposed reliability scores is able to consistently improve over supervised random forests and multi-output support vector regression. This is also true when the reliability threshold is selected automatically.

Performance Evaluation of Machine Learning Methods for Leaf Area Index Retrieval from Time-Series MODIS Reflectance Data.

Leaf area index (LAI) is an important biophysical parameter and the retrieval of LAI from remote sensing data is the only feasible method for generating LAI products at regional and global scales. However, most LAI retrieval methods use satellite observations at a specific time to retrieve LAI. Because of the impacts of clouds and aerosols, the LAI products generated by these methods are spatially incomplete and temporally discontinuous, and thus they cannot meet the needs of practical applications. To generate high-quality LAI products, four machine learning algorithms, including back-propagation neutral network (BPNN), radial basis function networks (RBFNs), general regression neutral networks (GRNNs), and multi-output support vector regression (MSVR) are proposed to retrieve LAI from time-series Moderate Resolution Imaging Spectroradiometer (MODIS) reflectance data in this study and performance of these machine learning algorithms is evaluated. The results demonstrated that GRNNs, RBFNs, and MSVR exhibited low sensitivity to training sample size, whereas BPNN had high sensitivity. The four algorithms performed slightly better with red, near infrared (NIR), and short wave infrared (SWIR) bands than red and NIR bands, and the results were significantly better than those obtained using single band reflectance data (red or NIR). Regardless of band composition, GRNNs performed better than the other three methods. Among the four algorithms, BPNN required the least training time, whereas MSVR needed the most for any sample size.

Interval Forecasting of Carbon Futures Prices Using a Novel Hybrid Approach with Exogenous Variables

This paper examines the interval forecasting of carbon futures prices in one of the most important carbon futures market. Specifically, the purpose of this study is to present a novel hybrid approach, which is composed of multioutput support vector regression (MSVR) and particle swarm optimization (PSO), in the task of forecasting the highest and lowest prices of carbon futures on the next trading day. Furthermore, we set out to investigate if considering some potential predictors, which have strong influence on carbon futures prices, in modeling process is useful for achieving better prediction performance. Aiming at testing its effectiveness, we benchmark the forecasting performance of our approach against four competitors. The daily interval prices of carbon futures contracts traded in the Intercontinental Futures Exchange from August 12, 2010, to November 13, 2014, are used as the experiment dataset. The statistical significance of the interval forecasts is examined. The proposed hybrid approach is found to demonstrate the higher forecasting performance relative to all other competitors. Our application offers practitioners a promising set of results with interval forecasting in carbon futures market.

Interval-valued time series forecasting using a novel hybrid HoltI and MSVR model

In view of the importance of interval-valued time series (ITS) modeling and forecasting, and the less research efforts made before, this study proposes an hybrid modeling framework combining interval Holt's exponential smoothing method (HoltI) and multi-output support vector regression (MSVR) for ITS forecasting. Following the philosophy of well-established hybrid “linear and nonlinear” modeling framework, HoltI and MSVR are committed to capture the linear and nonlinear patterns hidden in ITS, respectively. Different from the previous studies considering to model the highs and lows of intervals separately, the proposed hybrid method (termed as HoltI-MSVR) is used to model and forecast the daily highs and lows of ITS simultaneously, taking into account the possible interrelations between the bounds. Three ITS datasets extracted from finance market and energy market are used to compare the prediction performance of the HoltI-MSVR with five selected competitors. The experimental results are judged on the basis of statistical criteria, i.e., the goodness of forecast measure and the accuracy compared to competing forecasts test, and economic criteria, i.e., the returns obtained from a simple trading strategy based on the interval forecasts. The results obtained suggest that the proposed HoltI-MSVR is a promising alternative for ITS forecasting.

Estimation of the neuromodulation parameters from the planned volume of tissue activated in deep brain stimulation

Deep brain stimulation (DBS) is a therapy with promissory results for the treatment of movement disorders. It delivers electric stimulation via an electrode to a specific target brain region. The spatial extent of neural response to this stimulation is known as volume of tissue activated (VTA). Changes in stimulation parameters that control VTA, such as amplitude, pulse width and electrode configuration can affect the effectiveness of the DBS therapy. In this study, we develop a novel methodology for estimating suitable DBS neuromodulation parameters, from planned VTA, that attempts to maximize the therapeutic effects, and to minimize the adverse effects of DBS. For estimating the continuous outputs (amplitude and pulse width), we use multi-output support vector regression, taking the geometry of the VTA as input space. For estimating the electrode polarity configuration, we perform several classification problems, also using support vector machines from the same input space. Our methodology attains promising results for both the regression setting, and for predicting electrode active contacts and their polarity. Combining biological neural modeling techniques together with machine learning, we introduce a novel area of research where parameters of neuromodulation in DBS can be tuned by manually specifying a desired geometric volume.

Application of multi-output support vector regression on EMGs to decode hand continuous movement trajectory.

Applications of neural machine interfaces have received increased attention during the last decades. It is crucial to realize the continuous control of prosthetic devices based on biological signals. In order to deal with the highly nonlinear relationship between the Electromyography (EMG) signals and motion, this study presents a novel decoding approach which employs multi-output support vector regression (M-SVR). The proposed M-SVR is compared with other popular regression techniques and the experimental results demonstrate the effectiveness of M-SVR in hand continuous movement trajectory reconstruction.

Material behavior modeling with multi-output support vector regression

Based on neural network material-modeling technologies, a new paradigm, called multi-output support vector regression, is developed to model complex stress/strain behavior of materials. The constitutive information generally implicitly contained in the results of experiments, i.e., the relationships between stresses and strains, can be captured by training a support vector regression model within a unified architecture from experimental data. This model, inheriting the merits of the neural network based models, can be employed to model the behavior of modern, complex materials such as composites. Moreover, the architectures of the support vector regression built in this research can be more easily determined than that of the neural network. Therefore, the proposed constitutive models can be more conveniently applied to finite element analysis and other application fields.As an illustration, the behaviors of concrete in the state of plane stress under monotonic biaxial loading and compressive uniaxial cycle loading are modeled with the multi-output and single-output support regression respectively. The excellent results show that the support vector regression provides another effective approach for material modeling.

Mid-term interval load forecasting using multi-output support vector regression with a memetic algorithm for feature selection

Accurate forecasting of mid-term electricity load is an important issue for power system planning and operation. Instead of point load forecasting, this study aims to model and forecast mid-term interval loads up to one month in the form of interval-valued series consisting of both peak and valley points by using MSVR (Multi-output Support Vector Regression). In addition, an MA (Memetic Algorithm) based on the firefly algorithm is used to select proper input features among the feature candidates, which include time lagged loads as well as temperatures. The capability of this proposed interval load modeling and forecasting framework to predict daily interval electricity demands is tested through simulation experiments using real-world data from North America and Australia. Quantitative and comprehensive assessments are performed and the experimental results show that the proposed MSVR-MA forecasting framework may be a promising alternative for interval load forecasting.

Classification and Quantification of Binary Mixtures of Gases/Odors Using Thick-Film Gas Sensor Array Responses

This paper presents the results of experiments done for classification and quantification of two volatile organic compounds (VOCs), namely, acetone (CH <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> COCH <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> ) and 2-Propanol (CH <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> CHOHCH <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> ) in their single as well as in mixture forms. A sensor array consisting of four sensor elements was fabricated in our lab using thick-film fabrication technology. The steady-state responses of the sensor array were collected for the mentioned VOCs in their single as well as in mixture form. A hierarchical system consisting of gating network and three quantification networks was designed to classify and subsequently quantify the individual and mixture of VOCs. The classification accuracy results of gating network have been ensured using back-propagation neural network (BPNN) and support vector machine (SVM). For quantification multioutput support vector regression method was used in slave networks for single as well as binary mixture data. k-fold cross validation scheme was adopted for all the experiments. The average classification accuracy of gating network for the mixture data in raw form was 89.5% using BPNN and 94.7% using nu-SVM. With principal component analysis preprocessed data, the average accuracy was 95.6% with BPNN and 100% using nu-SVM, respectively. For quantification, good 0.9828 and 0.9764 correlation coefficients for the predicted versus real concentration of acetone and 2-Propanol, respectively, in mixture form were obtained. Thus, we report a promising approach for binary mixture of gases/odors analysis using thick-film gas sensor array responses.

A combination method for interval forecasting of agricultural commodity futures prices

Accurate interval forecasting of agricultural commodity futures prices over future horizons is challenging and of great interests to governments and investors, by providing a range of values rather than a point estimate. Following the well-established “linear and nonlinear” modeling framework, this study extends it to forecast interval-valued agricultural commodity futures prices with vector error correction model (VECM) and multi-output support vector regression (MSVR) (abbreviated as VECM–MSVR), which is capable of capturing the linear and nonlinear patterns exhibited in agricultural commodity futures prices. Two agricultural commodity futures prices from Chinese futures market are used to justify the performance of the proposed VECM–MSVR method against selected competitors. The quantitative and comprehensive assessments are performed and the results indicate that the proposed VECM–MSVR method is a promising alternative for forecasting interval-valued agricultural commodity futures prices.

Cloud removal of remote sensing image based on multi-output support vector regression

Removal of cloud cover on the satellite remote sensing image can effectively improve the availability of remote sensing images. For thin cloud cover, support vector value contourlet transform is used to achieve multi-scale decomposition of the area of thin cloud cover on remote sensing images. Through enhancing coefficients of high frequency and suppressing coefficients of low frequency, the thin cloud is removed. For thick cloud cover, if the areas of thick cloud cover on multi-source or multi-temporal remote sensing images do not overlap, the multi-output support vector regression learning method is used to remove this kind of thick clouds. If the thick cloud cover areas overlap, by using the multi-output learning of the surrounding areas to predict the surface features of the overlapped thick cloud cover areas, this kind of thick cloud is removed. Experimental results show that the proposed cloud removal method can effectively solve the problems of the cloud overlapping and radiation difference among multi-source images. The cloud removal image is clear and smooth.

Identification of Surrounding Rock Parameters Based on MSVR

An inversion method based on multi-output support vector regression (MSVR) is proposed for identifying the mechanical parameters of surrounding rock. This method considers the surrounding rock as a multi-output system during excavation, and the surveyed rock deformations of each monitoring section as its output. First, perform numerical experiments based on the principle of orthogonal test to obtain the calculated deformation values corresponding to different rock parameter combinations, and use them as the samples for training the model of MSVR as reflecting the nonlinear mapping relationship between rock and its deformations. Second, use the PSO to seek the optimal rock parameters based on measured deformations of rock mass. An example is employed to test the presented inversion method. The results showed that compared with the inversion method based on single-output support vector regression (SSVR), the proposed one is more inclined to reach the global optimization goals and achieve more reliable inversion results due to its consideration of the inherent correlativity among the measured deformations of each monitoring section.

Multiple-output support vector regression with a firefly algorithm for interval-valued stock price index forecasting

Highly accurate interval forecasting of a stock price index is fundamental to successfully making a profit when making investment decisions, by providing a range of values rather than a point estimate. In this study, we investigate the possibility of forecasting an interval-valued stock price index series over short and long horizons using multi-output support vector regression (MSVR). Furthermore, this study proposes a firefly algorithm (FA)-based approach, built on the established MSVR, for determining the parameters of MSVR (abbreviated as FA-MSVR). Three globally traded broad market indices are used to compare the performance of the proposed FA-MSVR method with selected counterparts. The quantitative and comprehensive assessments are performed on the basis of statistical criteria, economic criteria, and computational cost. In terms of statistical criteria, we compare the out-of-sample forecasting using goodness-of-forecast measures and testing approaches. In terms of economic criteria, we assess the relative forecast performance with a simple trading strategy. The results obtained in this study indicate that the proposed FA-MSVR method is a promising alternative for forecasting interval-valued financial time series.

Multioutput Support Vector Regression for Remote Sensing Biophysical Parameter Estimation

This letter proposes a multioutput support vector regression (M-SVR) method for the simultaneous estimation of different biophysical parameters from remote sensing images. General retrieval problems require multioutput (and potentially nonlinear) regression methods. M-SVR extends the single-output SVR to multiple outputs maintaining the advantages of a sparse and compact solution by using an e-insensitive cost function. The proposed M-SVR is evaluated in the estimation of chlorophyll content, leaf area index and fractional vegetation cover from a hyperspectral compact high-resolution imaging spectrometer images. The achieved improvement with respect to the single-output regression approach suggests that M-SVR can be considered a convenient alternative for nonparametric biophysical parameter estimation and model inversion.

Finite element model updating for large span spatial steel structure considering uncertainties

In order to establish the baseline finite element model for structural health monitoring, a new method of model updating was proposed after analyzing the uncertainties of measured data and the error of finite element model. In the new method, the finite element model was replaced by the multi-output support vector regression machine (MSVR). The interval variables of the measured frequency were sampled by Latin hypercube sampling method. The samples of frequency were regarded as the inputs of the trained MSVR. The outputs of MSVR were the target values of design parameters. The steel structure of National Aquatic Center for Beijing Olympic Games was introduced as a case for finite element model updating. The results show that the proposed method can avoid solving the problem of complicated calculation. Both the estimated values and associated uncertainties of the structure parameters can be obtained by the method. The static and dynamic characteristics of the updated finite element model are in good agreement with the measured data.