Evaluation Matrix for Smart Machine-Learning Algorithm Choice

Abstract

In Machine Learning, algorithm choice greatly affects the performance on a problem. Different advantages and disadvantages have to be taken into account in view of the specific use case. For critical applications, like in the medical field, model accuracy is paramount, while in the environment of embedded systems high computational efficiency may be of greater importance. Generally, expert knowledge is used to estimate a suitable algorithm for a defined application. However, the use of Machine Learning and Data Mining through non expert users is increasing.To simplify the process of algorithm choice for these inexperienced users, we propose an evaluation matrix for ranking machine learning algorithms, based on efficiency criteria. These criteria consist of different metrics, like accuracy, model complexity or scalability, which then in turn can be weighted for the underlying problem and evaluated to yield a use case specific algorithm ranking.Therefor efficiency criteria are discussed and assessment methods are defined. Resulting is a practicable evaluation matrix, which can be adapted to specific use cases through choice of importance weighting on the different efficiency criteria. The concept is of a modular nature, additional efficiency criteria can easily be included. Also, the set of algorithms considered in the analysis can be expanded through the established assessment methods.