Abstract

Support Vector Regression (SVR) is a powerful supervised machine learning model especially well suited to the normalized or binarized data. However, its quadratic complexity in the number of training examples eliminates it from training on large datasets, especially high dimensional with frequent retraining requirement. We propose a simple two-stage greedy selection of training data for SVR to maximize its validation set accuracy at the minimum number of training examples and illustrate the performance of such strategy in the context of Clash Royale Challenge 2019, concerned with efficient decks’ win rate prediction. Hundreds of thousands of labelled data examples were reduced to hundreds, optimized SVR was trained on to maximize the validation R2 score. The proposed model scored the first place in the Cash Royale 2019 challenge, outperforming over hundred of competitive teams from around the world.

Highlights

  • Support Vector Machine (SVM) is a supervised machine learning (ML) model developed as far back as in 1963 [1] on the basis of Vapnik-Chervonenkis computational theory of learning [2]

  • Support Vector Regression (SVR) extends the original capability of the SVM model into the regression space, while sharing the same model fundamental and properties as SVM does for classification: for instance in margin-maximizing hyper-plane characterization, tolerance of errors etc

  • High cost involved in computing large number of support vectors in SVR training process is a critical drawback compared to simpler supervised ML models, which unable to demonstrate such generalization ingenuity, are able to complete in a reasonable time: [9], [10], [11]

Read more

Summary

INTRODUCTION

Support Vector Machine (SVM) is a supervised machine learning (ML) model developed as far back as in 1963 [1] on the basis of Vapnik-Chervonenkis computational theory of learning [2]. High cost involved in computing large number of support vectors in SVR training process is a critical drawback compared to simpler supervised ML models, which unable to demonstrate such generalization ingenuity, are able to complete in a reasonable time: [9], [10], [11]. Based on the observation that a vast majority of the SVM (SVR) predictive power comes from fairly small number of key data-structure-capturing examples, an obvious attempt to eliminate huge computational cost of training SVR could be reduced by carefully selecting a small set of the critical training data points. In an attempt to address this challenge we have proposed a simple two-stage greedy search process that returns an ordered list of most predictive data points offering the most predictive SVR model based on incrementally added number of training examples.

COMPETITION DESCRIPTION
Data preparation
Hyperparameters’ setting
Greedy online backward-forward data selection
Greedy round-exhaustive forward data selection
Fine-tuning for further generalization improvements
EXPERIMENTAL RESULTS
CONCLUSIONS
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call