Parallel Portfolio Research Articles

Real-time solving of computationally hard problems using optimal algorithm portfolios

Various hard real-time systems have a desired requirement which is impossible to fulfill: to solve a computationally hard optimization problem within a short and fixed amount of time T, e.g., T = 0.5 seconds. For such a task, the exact, exponential algorithms, as well as various Polynomial-Time Approximation Schemes, are irrelevant because they can exceed T. What is left in practice is to combine various anytime algorithms in a parallel portfolio. The question is how to build such an optimal portfolio, given a budget of K computing cores. It is certainly not as simple as choosing the K best performing algorithms, because their results are possibly correlated (e.g., there is no point in choosing two good algorithm for the portfolio if they win on a similar set of instances). We prove that the decision variant of this problem is NP-complete, and furthermore that the optimization problem is approximable. On the practical side, our main contribution is a solution of the optimization problem of choosing K algorithms out of n, for a machine with K computing cores, and the related problem of detecting the minimum number of required cores to achieve an optimal portfolio, with respect to a given training set of instances. As a benchmark, we took instances of a hard optimization problem that is prevalent in the real-time industry, in which the challenge is to decide on the best action within time T. We include the results of numerous experiments that compare the various methods. Hence, a side effect of our tests is that it gives the first systematic empirical evaluation of the relative success of various known stochastic-search algorithms in coping with a hard combinatorial optimization problems under a very short and fixed timeout.

Heterogeneous Island Models and Their Application to Recommender Systems and Electric Vehicle Charging

In this paper we describe a general framework for parallel optimization based on the island model of evolutionary algorithms. The framework runs a number of optimization methods in parallel with periodic communication. In this way, it essentially creates a parallel ensemble of optimization methods. At the same time, the system contains a planner that decides which of the available optimization methods should be used to solve the given optimization problem and changes the distribution of such methods during the run of the optimization. Thus, the system effectively solves the problem of online parallel portfolio selection. The proposed system is evaluated in a number of common benchmarks with various problem encodings as well as in two real-life problems — the optimization in recommender systems and the training of neural networks for the control of electric vehicle charging.

Generative Adversarial Construction of Parallel Portfolios.

Since automatic algorithm configuration methods have been very effective, recently there is increasing research interest in utilizing them for automatic solver construction, resulting in several notable approaches. For these approaches, a basic assumption is that the given training set could sufficiently represent the target use cases such that the constructed solvers can generalize well. However, such an assumption does not always hold in practice since in some cases, we might only have scarce and biased training data. This article studies effective construction approaches for the parallel algorithm portfolios that are less affected in these cases. Unlike previous approaches, the proposed approach simultaneously considers instance generation and portfolio construction in an adversarial process, in which the aim of the former is to generate instances that are challenging for the current portfolio, while the aim of the latter is to find a new component solver for the portfolio to better solve the newly generated instances. Applied to two widely studied problem domains, that is, the Boolean satisfiability problems (SAT) and the traveling salesman problems (TSPs), the proposed approach identified parallel portfolios with much better generalization than the ones generated by the existing approaches when the training data were scarce and biased. Moreover, it was further demonstrated that the generated portfolios could even rival the state-of-the-art manually designed parallel solvers.

Automatic Construction of Parallel Portfolios via Explicit Instance Grouping

Exploiting parallelism is becoming more and more important in designing efficient solvers for computationally hard problems. However, manually building parallel solvers typically requires considerable domain knowledge and plenty of human effort. As an alternative, automatic construction of parallel portfolios (ACPP) aims at automatically building effective parallel portfolios based on a given problem instance set and a given rich configuration space. One promising way to solve the ACPP problem is to explicitly group the instances into different subsets and promote a component solver to handle each of them. This paper investigates solving ACPP from this perspective, and especially studies how to obtain a good instance grouping. The experimental results on two widely studied problem domains, the boolean satisfiability problems (SAT) and the traveling salesman problems (TSP), showed that the parallel portfolios constructed by the proposed method could achieve consistently superior performances to the ones constructed by the state-of-the-art ACPP methods, and could even rival sophisticated hand-designed parallel solvers.

Parallel algorithm portfolios with performance forecasting

ABSTRACTWe propose a novel algorithm portfolio model that incorporates time series forecasting techniques to predict online the performance of its constituent algorithms. The predictions are used to allocate computational resources to the algorithms, accordingly. The proposed model is demonstrated on parallel algorithm portfolios consisting of three popular metaheuristics, namely tabu search, variable neighbourhood search, and multistart local search. Moving average and exponential smoothing techniques are employed for forecasting purposes. A challenging combinatorial problem, namely the detection of circulant weighing matrices, is selected as the testbed for the analysis of the proposed approach. Experimental evidence and statistical analysis provide insight on the performance of the proposed algorithms and reveal the benefits of using forecasting techniques for resource allocation in algorithm portfolios.

A parallel portfolio approach to configuration optimization for large software product lines

SummarySoftware product line (SPL) engineering demands for optimal or near‐optimal products that balance multiple often competing and conflicting objectives. A major challenge for large SPLs is to efficiently explore a huge space of various products and satisfy a large number of predefined constraints simultaneously. To improve the optimality and convergence speed, we propose a parallel portfolio approach, called IBEAPORT, which designs three algorithm variants by incorporating constraint solving into the indicator‐based evolutionary algorithm in different ways and performs these variants by utilizing parallelization techniques. Our approach utilizes the exploration capabilities of different algorithms and improves optimality as far as possible within a limited time budget. We evaluate our approach on five large‐scale real‐world SPLs. Empirical results demonstrate that our approach significantly outperforms the state of the art for all five SPLs on a quality indicator and a diversity indicator. Moreover, IBEAPORT quickly converges to a relatively stable hypervolume value even for the largest SPL with 6888 features.

Parallel population-based algorithm portfolios: An empirical study

Although many algorithms have been proposed, no single algorithm is better than others on all types of problems. Therefore, the search characteristics of different algorithms that show complementary behavior can be combined through portfolio structures to improve the performance on a wider set of problems. In this work, a portfolio of the Artificial Bee Colony, Differential Evolution and Particle Swarm Optimization algorithms was constructed and the first parallel implementation of the population-based algorithm portfolio was carried out by means of a Message Passing Interface environment. The parallel implementation of an algorithm or a portfolio can be performed by different models such as master-slave, coarse-grained or a hybrid of both, as used in this study. Hence, the efficiency and running time of various parallel implementations with different parameter values and combinations were investigated on benchmark problems. The performance of the parallel portfolio was compared to those of the single constituent algorithms. The results showed that the proposed models reduced the running time and the portfolio delivered a robust performance compared to each constituent algorithm. It is observed that the speedup gained over the sequential counterpart changed significantly depending on the structure of the portfolio. The portfolio is also applied to a training of neural networks which has been used for time series prediction. Result demonstrate that, portfolio is able to produce good prediction accuracy.

Massively parallel processing of recursive multi-period portfolio models

A recursive portfolio decision system is extended with parallel processing capability monitored by the Genetic Hybrid Algorithm (GHA). Massively parallel portfolio efficiency testing is conducted using stochastic simulation. Genuine out-of-sample forecasts are generated for all titles in the universe using fast cutting-edge time series algorithms. The computation of dynamic optimal portfolio weights is done within an affine multi-period setting. The terminal wealth within the planning horizon forms a moving target as the system evolves through time and only current transactions are carried out. Fixed and variable transaction costs are recognized without increasing computational complexity. We show using a recursive multi-period portfolio framework (RMP) that robust grid search and stochastic simulation with thousands of parallel processors can be conducted to provide evidence on portfolio efficiency. The downside risk of the RMP-strategy is significantly lower than that of the corresponding buy-and-hold strategy. The upside potential of RMP is much better than that of buy-and-hold. The non-parametric test procedure is independent of the underlying model and hence completely general. The modular structure of the system allows new forecasting techniques and optimization formulations to be introduced and tested in future development efforts.

Parallelizing simplex within SMT solvers

The usual approach in parallelizing SAT and SMT solvers is either to explore different parts of the search space in parallel (divide-and-conquer approach) or to run multiple instances of the same solver with suitably altered parameters in parallel, possibly exchanging some information during the solving process (parallel portfolio approach). Quite a different approach is to parallelize the execution of time-consuming algorithms that check for satisfiability and propagations during the search space exploration. Since most of the execution time is spent in these procedures, their efficient parallelization might be a promising research direction. In this paper we present our experience in parallelizing the simplex algorithm which is typically used in the SMT context to check the satisfiability of linear arithmetic constraints. We provide a detailed description of this approach and present experimental results that evaluate the potential of the approach compared to the parallel portfolio approach. We also consider the combination of the two approaches.

Automatic construction of parallel portfolios via algorithm configuration

Since 2004, increases in computational power described by Moore's law have substantially been realized in the form of additional cores rather than through faster clock speeds. To make effective use of modern hardware when solving hard computational problems, it is therefore necessary to employ parallel solution strategies. In this work, we demonstrate how effective parallel solvers for propositional satisfiability (SAT), one of the most widely studied NP-complete problems, can be produced automatically from any existing sequential, highly parametric SAT solver. Our Automatic Construction of Parallel Portfolios (ACPP) approach uses an automatic algorithm configuration procedure to identify a set of configurations that perform well when executed in parallel. Applied to two prominent SAT solvers, Lingeling and clasp, our ACPP procedure identified 8-core solvers that significantly outperformed their sequential counterparts on a diverse set of instances from the application and hard combinatorial category of the 2012 SAT Challenge. We further extended our ACPP approach to produce parallel portfolio solvers consisting of several different solvers by combining their configuration spaces. Applied to the component solvers of the 2012 SAT Challenge gold medal winning SAT Solver pfolioUZK, our ACPP procedures produced a significantly better-performing parallel SAT solver.

Recursive optimization on converting CNF to DNF using grid computing

A NP hard problem CNF to DNF conversion is a vast area of research for AI, circuit design, FPGA’s (Miltersen et al. in On converting CNF To DNF, 2003), PLA’s, etc. (Beame in A switching lemma primer, 1994; Kottler and Kaufmann in SArTagnan—a parallel portfolio SAT solver with lockless physical clause sharing, 2011). Optimization and its statistics has become a potential requirement for analysis and behavior of normal form conversion. Various applications are in its requirement like gnome analysis, grid computing, bioinformatics, imaging systems, rough sets require higher variable processing algorithm. Problem statement is—design and implementation of optimal conjunctive normal form to optimal (prime implicants) disjunctive normal form conversion which is an “NP hard problem conversion to an NP complete”. Thus CNF to DNF can only be considered to evaluate best performance for higher variable processing on high end systems. The best-known representations of Boolean functions f are those as disjunctions of terms (DNFs) and as conjunctions of clauses (CNFs) (Beame 1994; Kottler and Kaufmann 2011) (Wegener in The complexity of boolean functions, 1987). It is convenient to define the DNF size of f as the minimal number of terms in a DNF representing f and the CNF size as the minimal number of clauses in a CNF representing f (Kottler and Kaufmann 2011).

Deterministic Parallel DPLL

Current parallel SAT solvers suffer from a non-deterministic behavior. This is the consequence of their architectures which rely on weak synchronizing in an attempt to maximize performance. This behavior is a clear downside for practitioners, who are used to both runtime and solution reproducibility. In this paper, we propose the first Deterministic Parallel DPLL engine. Our experimental results clearly show that our approach preserves the performance of the parallel portfolio approach while ensuring full reproducibility of the results.

Learning parallel portfolios of algorithms

A wide range of combinatorial optimization algorithms have been developed for complex reasoning tasks. Frequently, no single algorithm outperforms all the others. This has raised interest in leveraging the performance of a collection of algorithms to improve performance. We show how to accomplish this using a Parallel Portfolio of Algorithms (PPA). A PPA is a collection of diverse algorithms for solving a single problem, all running concurrently on a single processor until a solution is produced. The performance of the portfolio may be controlled by assigning different shares of processor time to each algorithm. We present an effective method for finding a PPA in which the share of processor time allocated to each algorithm is fixed. Finding the optimal static schedule is shown to be an NP-complete problem for a general class of utility functions. We present bounds on the performance of the PPA over random instances and evaluate the performance empirically on a collection of 23 state-of-the-art SAT algorithms. The results show significant performance gains over the fastest individual algorithm in the collection.