Abstract
This paper proposes a new multi-kernel learning ensemble algorithm, called Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR, which can be regarded as an extension of multi-kernel learning (MKL) and weighted support vector regression (WSVR). The first novelty is to add the <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula> norm of the weights of the combined kernel function to the objective function of WSVR, which is used to adaptively select the optimal base models and their parameters. In addition, an accelerated method based on fast iterative shrinkage thresholding algorithm (FISTA) is developed to solve the weights of the combined kernel function. The second novelty is to propose an integrated learning framework based on AdaBoost, named Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR. In this framework, we integrate FISTA into AdaBoost. At each iteration, we optimize the weights of the combined kernel function and update the weights of the training samples at the same time. Then an ensemble regression function of a set of regression functions is output. Finally, two groups of the experiments are designed to verify the performance of our algorithm. On the first group of the experiments including eight datasets from UCI machine learning repository, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 11.14% and 9.08% on average, respectively. Furthermore, on the second group of the experiments including the COVID-19 epidemic datasets from eight countries, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 31.19% and 29.98% on average, respectively.
Highlights
Support vector machine (SVM) [1], [2] is an algorithm based on supervised learning mode, which can be used for data classification, model recognition and regression analysis
METHODOLOGY we fisrt introduce the key idea of L1MKLSVR, which is the basis of our algorithm, we provide the details of L1MKL-weighted support vector regression (WSVR) and Ada-L1MKL-WSVR, respectively
To learn the weights of the training samples and the combined kernel function simultaneously, we introduce the weights λ of the training samples into L1MKL-Support vector regression (SVR), namely L1MKLWSVR, which can be expressed as the following optimization problem, i.e., 1S min w,D,b,ξ,ξ∗ 2 s=1 ws ds
Summary
Support vector machine (SVM) [1], [2] is an algorithm based on supervised learning mode, which can be used for data classification, model recognition and regression analysis. It has a strong mathematical foundation and theoretical support. SVR has many advantages in solving small sample, nonlinear and high dimensional pattern recognition, and has been widely applied to practical problems, including traffic velocity prediction [8], conductivity prediction [9], spatial prediction of landslide susceptibility [10], and stock price forecasting [11]. Using MKL instead of the traditional single-kernel learning can greatly improve the interpretability and generalization performance of the model [13]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
