Online Algorithm Selection Research Articles

MAB-OS: Multi-Armed Bandits Metaheuristic Optimizer Selection

Metaheuristic algorithms are derivative-free optimizers designed to estimate the global optima for optimization problems. Keeping balance between exploitation and exploration and the performance complementarity between the algorithms have led to the introduction of quite a few metaheuristic methods. In this work, we propose a framework based on Multi-Armed Bandits (MAB) problem, which is a classical Reinforcement Learning (RL) method, to intelligently select a suitable optimizer for each optimization problem during the optimization process. This online algorithm selection technique leverages on the convergence behavior of the algorithms to find the right balance of exploration–exploitation by choosing the update rule of the algorithm with the most estimated improvement in the solution. By performing experiments with three armed-bandits being Harris Hawks Optimizer (HHO), Differential Evolution (DE), and Whale Optimization Algorithm (WOA), we show that the MAB Optimizer Selection (named as MAB-OS) framework has the best overall performance on different types of fitness landscapes in terms of both convergence rate and the final solution. The data and codes used for this work are available at: https://github.com/BaratiLab/MAB-OS.

Machine Learning for Online Algorithm Selection under Censored Feedback

In online algorithm selection (OAS), instances of an algorithmic problem class are presented to an agent one after another, and the agent has to quickly select a presumably best algorithm from a fixed set of candidate algorithms. For decision problems such as satisfiability (SAT), quality typically refers to the algorithm's runtime. As the latter is known to exhibit a heavy-tail distribution, an algorithm is normally stopped when exceeding a predefined upper time limit. As a consequence, machine learning methods used to optimize an algorithm selection strategy in a data-driven manner need to deal with right-censored samples, a problem that has received little attention in the literature so far. In this work, we revisit multi-armed bandit algorithms for OAS and discuss their capability of dealing with the problem. Moreover, we adapt them towards runtime-oriented losses, allowing for partially censored data while keeping a space- and time-complexity independent of the time horizon. In an extensive experimental evaluation on an adapted version of the ASlib benchmark, we demonstrate that theoretically well-founded methods based on Thompson sampling perform specifically strong and improve in comparison to existing methods.

A Regression-Based Methodology for Online Algorithm Selection

Algorithm selection approaches have achieved impressive performance improvements in many areas of AI. Most of the literature considers the offline algorithm selection problem, where the initial selection model is never updated after training. However, new data from running algorithms on instances becomes available when algorithms are selected and run. We investigate how this online data can be used to improve the selection model over time. This is especially relevant when insufficient training instances were used, but potentially improves the performance of algorithm selection in all cases. We formally define the online algorithm selection problem and model it as a contextual multi-armed bandit problem, propose a methodology for solving it, and empirically demonstrate performance improvements. We also show that our online algorithm selection method can be used when no training data whatsoever is available, a setting where offline algorithm selection cannot be used. Our experiments indicate that a simple greedy approach achieves the best performance.

Non-stationary multiple-antenna interference cancellation for unsynchronized OFDM systems with distributed training

Adaptive interference cancellation is addressed in an asynchronous orthogonal frequency-division multiplexing (OFDM) system with a frame containing a number of temporally distributed pilot symbols, e.g., as in the IEEE 802.16-2004 standard. Non-stationary symbol-by-symbol switching technique is proposed that is based on banks of the second-order statistics semi-blind solutions with on-line algorithm selection using a higher-order statistics criterion. Three banks of algorithms are formulated and analyzed by means of comparison to the conventional stationary pilot-based solution and to the non-asymptotic maximum-likelihood benchmark especially developed for assessment of the proposed interference cancellation algorithms. Their efficiency is compared to the conventional stationary solutions by means of simulations in a WiMAX (worldwide interoperability for microwave access) environment.

Learning dynamic algorithm portfolios

Algorithm selection can be performed using a model of runtime distribution, learned during a preliminary training phase. There is a trade-off between the performance of model-based algorithm selection, and the cost of learning the model. In this paper, we treat this trade-off in the context of bandit problems. We propose a fully dynamic and online algorithm selection technique, with no separate training phase: all candidate algorithms are run in parallel, while a model incrementally learns their runtime distributions. A redundant set of time allocators uses the partially trained model to propose machine time shares for the algorithms. A bandit problem solver mixes the model-based shares with a uniform share, gradually increasing the impact of the best time allocators as the model improves. We present experiments with a set of SAT solvers on a mixed SAT-UNSAT benchmark; and with a set of solvers for the Auction Winner Determination problem.

GAUSS: an online algorithm selection system for numerical quadrature

We describe the design and implementation of GAUSS — an online algorithm selection system for numerical quadrature. Given a quadrature problem and performance constraints on its solution, GAUSS selects the best (or nearly best) algorithm. GAUSS uses inductive logic programming to generalize a database of performance data; this produces high-level rules that correlate problem features with algorithm performance. Such rules then serve as the basis for recommending algorithms for new problem instances. GAUSS functions online (new data and information can be incrementally incorporated) and can also provide phenomenological explanations of algorithm recommendations.

Fuzzy 意思決定を用いたアルゴリズム選択の一手法

Generally, various approaches, like optimization, stabilization, adaptation and so on, can be taken to solve a control problem. Thus, there exist several kinds of algorithms to solve a certain problem and each algorithm's characteristics depend on the approach to be taken. In the case that these algorithms are applied to practical system problems, it is necessary to examine the algorithms' effectiveness and select a suitable one among usable algorithms.On the other hand, in a man-machine system, a human operator occasionally selects the better one by use of his experimental know-how or a priori information. But it is very difficult for the operator to make a good decision continuously in a real time system.In this paper, an on-line algorithm selection problem is considered. Firstly, the algorithm selection process is formulated as a fuzzy decision-making problem and a fuzzy algorithm selection method is proposed. If-then type rules are used to reason the effectiveness of the algorithms. A priori information and know-how is utilized in the reasoning rules. Secondly, this method is applied to the guidance and control problem in a command and control system. Finally, simulation is conducted to evaluate the effectiveness of the proposed method.