Learning the Optimal Representation Dimension for Restricted Boltzmann Machines

Abstract

Hyperparameters refer to a set of parameters of a machine learning model that are fixed and not adjusted during training. A fundamental problem in this context is hyperparameter tuning which refers to the problem of identifying the best values for a set of model hyperparameters for a given task. In particular, model performance strongly depends on the choice of hyperparameters, and the right choice often determines the difference between average and stateof- the-art performance. This becomes especially important in models with many hyperparameters, as is common in deep learning models (DL) and automated machine learning (AutoML). However, finding the best set of hyperparameters for a model faced with a given task is very difficult in general, given the large state space and the high computational cost of assessing the quality of a given set of hyperparameters.