Stochastic Schemata Exploiter-Based AutoML

Abstract

Machine Learning (ML) models are often determined by relying on knowledge and experience. In recent years, many automatic building methods are proposed, but there are some problems related to accuracy, computation cost, and explainability. We propose the Stochastic Schemata Exploiter-Based AutoML. The Stochastic Schemata Exploiter (SSE) is one of the Evolutionary Algorithms. In each generation, SSE calculates fitness of the individuals, defines the best subsets according to their average fitness, generates individuals based on the subsets, and applies a mutation operator. Since the original SSE uses a binary string representation, we have to modify the SSE algorithm in the following points for parameter optimization: the initialization method, the schema extraction method, the new individual generation method, the mutation method, and the generation update method. In this paper, we propose a genetic representation of the stacking model and optimize the stacking model using SSE. Compared with the Genetic Algorithm, the Tree-structured Parzen Estimator, the Covariance Matrix Adaptation - Evolution Strategy, and Random Search, SSE shows an interesting feature: a better accuracy for combinatorial optimization problems with categorical, discrete, and continuous variables, such as hyper-parameter optimization. In addition, we propose the visualization of the process of SSE. The visualization helps us to understand the process, which is another advantage of the SSE-based optimization (SSEopt).