Guest editorial: model selection and optimization in machine learning

Abstract

Machine learning methods largely benefit from optimization techniques in order to find an optimal model for future predictions and decisions. The interplay of machine learning and optimization methods is much like operations research (OR). Optimization, also called mathematical programming, is a subfield of OR. Both machine learning and OR are concerned with modeling of systems related to real-world problems. In machine learning (ML), the common practice is to use classical optimization techniques. However, due to massive and large-scale data sets faced in real world problems, optimization becomes a challenging task and traditional approaches cannot keep up with expectations. Accordingly, optimization methods adapted or integrated for machine learning tasks are needed to make ML more feasible for real world data sets. Another important challenging task is model selection. Because of the mathematical structure of the optimization model, there are parameters to be searched offline for the training data. Statistical model selection methods such as crossvalidation can be very time consuming when the size and the dimension of the training data is large. Thus, developing powerful model selection methods is an important factor for the feasibility of the algorithm solving the optimization problem. This special issue on “Model Selection and Optimization in ML” was inspired from the stream “Model Selection and Optimization Techniques in Machine Learning” organized by Kristiaan Pelckmans and Sureyya Ozogur-Akyuz at 23rd European Conference on Operational Research held in Bonn, Germany, July 5–8, 2009.