Algorithm Selection System Research Articles

Which algorithm to select in sports timetabling?

Any sports competition needs a timetable, specifying when and where teams meet each other. The recent International Timetabling Competition (ITC2021) on sports timetabling showed that, although it is possible to develop general algorithms, the performance of each algorithm varies considerably over the problem instances. This paper provides a problem type analysis for sports timetabling, resulting in powerful insights into the strengths and weaknesses of eight state-of-the-art algorithms. Based on machine learning techniques, we propose an algorithm selection system that predicts which algorithm is likely to perform best based on the type of competition and constraints being used (i.e., the problem type) in a given sports timetabling problem instance. Furthermore, we visualize how the problem type relates to algorithm performance, providing insights and possibilities to further enhance several algorithms. Finally, we assess the empirical hardness of the instances. Our results are based on large computational experiments involving about 50 years of CPU time on more than 500 newly generated problem instances.

Algorithm selection for protein–ligand docking: strategies and analysis on ACE

The present study investigates the use of algorithm selection for automatically choosing an algorithm for any given protein–ligand docking task. In drug discovery and design process, conceptualizing protein–ligand binding is a major problem. Targeting this problem through computational methods is beneficial in order to substantially reduce the resource and time requirements for the overall drug development process. One way of addressing protein–ligand docking is to model it as a search and optimization problem. There have been a variety of algorithmic solutions in this respect. However, there is no ultimate algorithm that can efficiently tackle this problem, both in terms of protein–ligand docking quality and speed. This argument motivates devising new algorithms, tailored to the particular protein–ligand docking scenarios. To this end, this paper reports a machine learning-based approach for improved and robust docking performance. The proposed set-up is fully automated, operating without any expert opinion or involvement both on the problem and algorithm aspects. As a case study, an empirical analysis was performed on a well-known protein, Human Angiotensin-Converting Enzyme (ACE), with 1428 ligands. For general applicability, AutoDock 4.2 was used as the docking platform. The candidate algorithms are also taken from AutoDock 4.2. Twenty-eight distinctly configured Lamarckian-Genetic Algorithm (LGA) are chosen to build an algorithm set. ALORS which is a recommender system-based algorithm selection system was preferred for automating the selection from those LGA variants on a per-instance basis. For realizing this selection automation, molecular descriptors and substructure fingerprints were employed as the features characterizing each target protein–ligand docking instance. The computational results revealed that algorithm selection outperforms all those candidate algorithms. Further assessment is reported on the algorithms space, discussing the contributions of LGA’s parameters. As it pertains to protein–ligand docking, the contributions of the aforementioned features are examined, which shed light on the critical features affecting the docking performance.

Benchmarking Feature-Based Algorithm Selection Systems for Black-Box Numerical Optimization

Feature-based algorithm selection aims to automatically find the best one from a portfolio of optimization algorithms on an unseen problem based on its landscape features. Feature-based algorithm selection has recently received attention in the research field of black-box numerical optimization. However, there is still room for the analysis of algorithm selection for black-box optimization. Most previous studies have focused only on whether an algorithm selection system can outperform the single-best solver (SBS) in a portfolio. In addition, a benchmarking methodology for algorithm selection systems has not been well investigated in the literature. In this context, this article analyzes algorithm selection systems on the 24 noiseless black-box optimization benchmarking functions. First, we demonstrate that the first successful performance measure is more reliable than the expected runtime measure for benchmarking algorithm selection systems. Then, we examine the influence of randomness on the performance of algorithm selection systems. We also show that the performance of algorithm selection systems can be significantly improved by using sequential least squares programming as a presolver. We point out that the difficulty of outperforming the SBS depends on algorithm portfolios, cross-validation methods, and dimensions. Finally, we demonstrate that the effectiveness of algorithm portfolios depends on various factors. These findings provide fundamental insights for algorithm selection for black-box optimization.

Understanding and improving ontology reasoning efficiency through learning and ranking

Ontologies are the fundamental building blocks of the Semantic Web and Linked Data. Reasoning is critical to ensure the logical consistency of ontologies, and to compute inferred knowledge from an ontology. It has been shown both theoretically and empirically that, despite decades of intensive work on optimising ontology reasoning algorithms, performing core reasoning tasks on large and expressive ontologies is time-consuming and resource-intensive. In this paper, we present the meta-reasoning framework R2O2* to tackle the important problems of understanding the source of TBox reasoning hardness and predicting and optimising TBox reasoning efficiency by exploiting machine learning techniques. R2O2* combines state-of-the-art OWL 2 DL reasoners as well as an efficient OWL 2 EL reasoner as components, and predicts the most efficient one by using an ensemble of robust learning algorithms including XGBoost and Random Forests. A comprehensive evaluation on a large and carefully curated ontology corpus shows that R2O2* outperforms all six component reasoners as well as AutoFolio, a robust and strong algorithm selection system.

Automated Algorithm Selection: Survey and Perspectives.

It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single algorithm defines the state of the art; instead, there is a set of algorithms with complementary strengths. This performance complementarity can be exploited in various ways, one of which is based on the idea of selecting, from a set of given algorithms, for each problem instance to be solved the one expected to perform best. The task of automatically selecting an algorithm from a given set is known as the per-instance algorithm selection problem and has been intensely studied over the past 15 years, leading to major improvements in the state of the art in solving a growing number of discrete combinatorial problems, including propositional satisfiability and AI planning. Per-instance algorithm selection also shows much promise for boosting performance in solving continuous and mixed discrete/continuous optimisation problems. This survey provides an overview of research in automated algorithm selection, ranging from early and seminal works to recent and promising application areas. Different from earlier work, it covers applications to discrete and continuous problems, and discusses algorithm selection in context with conceptually related approaches, such as algorithm configuration, scheduling, or portfolio selection. Since informative and cheaply computable problem instance features provide the basis for effective per-instance algorithm selection systems, we also provide an overview of such features for discrete and continuous problems. Finally, we provide perspectives on future work in the area and discuss a number of open research challenges.

On Algorithm Selection, with an application to combinatorial search problems

The Algorithm Selection Problem is to select the most appropriate way for solving a problem given a choice of different ways. Some of the most prominent and successful applications come from Artificial Intelligence and in particular combinatorial search problems. Machine Learning has established itself as the de facto way of tackling the Algorithm Selection Problem. Yet even after a decade of intensive research, there are no established guidelines as to what kind of Machine Learning to use and how. This dissertation presents an overview of the field of Algorithm Selection and associated research and highlights the fundamental questions left open and problems facing practitioners. In a series of case studies, it underlines the difficulty of doing Algorithm Selection in practice and tackles issues related to this. The case studies apply Algorithm Selection techniques to new problem domains and show how to achieve significant performance improvements. Lazy learning in constraint solving and the implementation of the all different constraint are the areas in which we improve on the performance of current state of the art systems. The case studies furthermore provide empirical evidence for the effectiveness of using the misclassification penalty as an input to Machine Learning. After having established the difficulty, we present an effective technique for reducing it. Machine Learning ensembles are a way of reducing the background knowledge and experimentation required from the researcher while increasing the robustness of the system. Ensembles do not only decrease the difficulty, but can also increase the performance of Algorithm Selection systems. They are used to much the same ends in Machine Learning itself. We finally tackle one of the great remaining challenges of Algorithm Selection – which Machine Learning technique to use in practice. Through a large-scale empirical evaluation Constraints (2015) 20:481–482 DOI 10.1007/s10601-015-9214-x * Lars Kotthoff larsko@cs.ubc.ca 1 BETA Lab, Department of Computer Science, University of British Columbia, Vancouver, BC, Canada on diverse data taken from Algorithm Selection applications in the literature, we establish recommendations for Machine Learning algorithms that are likely to perform well in Algorithm Selection for combinatorial search problems. The recommendations are based on strong empirical evidence and additional statistical simulations. The research presented in this dissertation significantly reduces the knowledge threshold for researchers who want to perform Algorithm Selection in practice. It makes major contributions to the field of Algorithm Selection by investigating fundamental issues that have been largely ignored by the research community so far. School: University of St Andrews Supervisors: Ian Miguel Ian Gent Graduated: Wednesday, June 20, 2012 Link to full text: http://www.a4cp.org/sites/default/files/lars_kotthoff_-_on_algorithm_ selection_with_an_application_to_combinatorial_search_problems.pdf 482 Constraints (2015) 20:481–482

Algorithm Selection for Combinatorial Search Problems: A Survey

The algorithm selection problem is concerned with selecting the best algorithm to solve a given problem instance on a case‐by‐case basis. It has become especially relevant in the last decade, with researchers increasingly investigating how to identify the most suitable existing algorithm for solving a problem instance instead of developing new algorithms. This survey presents an overview of this work focusing on the contributions made in the area of combinatorial search problems, where algorithm selection techniques have achieved significant performance improvements. We unify and organise the vast literature according to criteria that determine algorithm selection systems in practice. The comprehensive classification of approaches identifies and analyzes the different directions from which algorithm selection has been approached. This article contrasts and compares different methods for solving the problem as well as ways of using these solutions.

A machine learning approach to algorithm selection for $\mathcal{NP}$ -hard optimization problems: a case study on the MPE problem

Given one instance of an \(\mathcal{NP}\) -hard optimization problem, can we tell in advance whether it is exactly solvable or not? If it is not, can we predict which approximate algorithm is the best to solve it? Since the behavior of most approximate, randomized, and heuristic search algorithms for \(\mathcal{NP}\) -hard problems is usually very difficult to characterize analytically, researchers have turned to experimental methods in order to answer these questions. In this paper we present a machine learning-based approach to address the above questions. Models induced from algorithmic performance data can represent the knowledge of how algorithmic performance depends on some easy-to-compute problem instance characteristics. Using these models, we can estimate approximately whether an input instance is exactly solvable or not. Furthermore, when it is classified as exactly unsolvable, we can select the best approximate algorithm for it among a list of candidates. In this paper we use the MPE (most probable explanation) problem in probabilistic inference as a case study to validate the proposed methodology. Our experimental results show that the machine learning-based algorithm selection system can integrate both exact and inexact algorithms and provide the best overall performance comparing to any single candidate algorithm.

Learning-based algorithm selection for image segmentation

Segmentation of nontrivial images is one of the most important tasks in image processing. It is easy for human being, but extremely difficult for computers. With the purpose of finding optimal segmentation algorithm for every image through learning from human experience, this paper investigates the manual segmentation process and thus presents a performance prediction based algorithm selection model to bridge the knowledge gap between images and segmentation algorithms. Derived from that model, a framework of learning-based algorithm selection system is proposed to automatically segment all images in a large database. A simulation system is designed to select the optimal segmentation algorithm from four candidates for synthetic images. The system is tested on 9000 images by comparing with the manual algorithm selection. The best algorithms are selected for 85% of the cases. If we also regard the second best algorithm as acceptable, more than 97% of images can be properly segmented. The satisfied result demonstrated that this study has provided a promising approach to achieve automated image segmentation.

GAUSS: an online algorithm selection system for numerical quadrature

We describe the design and implementation of GAUSS — an online algorithm selection system for numerical quadrature. Given a quadrature problem and performance constraints on its solution, GAUSS selects the best (or nearly best) algorithm. GAUSS uses inductive logic programming to generalize a database of performance data; this produces high-level rules that correlate problem features with algorithm performance. Such rules then serve as the basis for recommending algorithms for new problem instances. GAUSS functions online (new data and information can be incrementally incorporated) and can also provide phenomenological explanations of algorithm recommendations.