General-purpose Heuristics Research Articles

Anytime Heuristic and Monte Carlo Methods for Large-Scale Simultaneous Coalition Structure Generation and Assignment

Optimal simultaneous coalition structure generation and assignment is computationally hard. The state-of-the-art can only compute solutions to problems with severely limited input sizes, and no effective approximation algorithms that are guaranteed to yield high-quality solutions are expected to exist. Real-world optimization problems, however, are often characterized by large-scale inputs and the need for generating feasible solutions of high quality in limited time. In light of this, and to make it possible to generate better feasible solutions for difficult large-scale problems efficiently, we present and benchmark several different anytime algorithms that use general-purpose heuristics and Monte Carlo techniques to guide search. We evaluate our methods using synthetic problem sets of varying distribution and complexity. Our results show that the presented algorithms are superior to previous methods at quickly generating near-optimal solutions for small-scale problems, and greatly superior for efficiently finding high-quality solutions for large-scale problems. For example, for problems with a thousand agents and values generated with a uniform distribution, our best approach generates solutions 99.5% of the expected optimal within seconds. For these problems, the state-of-the-art solvers fail to find any feasible solutions at all.

Meta-Lamarckian learning in multi-objective optimization for mobile social network search

Mobile Social Networks (MSNs) have recently brought a revolution in socially-oriented applications and services for mobile phones. In this paper, we consider the search problem in a MSN that aims at simultaneously maximizing the user's search outcome (recall) and mobile phone performance (battery usage). Because of the conflicting nature of these two objectives, the problem is dealt within the context of Multi-Objective Optimization (MOO). Our proposed approach hybridizes a Multi-objective Evolutionary Algorithm based on Decomposition (MOEA/D) with a Meta-Lamarckian (ML) learning strategy that learns from the problem's properties and objective functions. The ML strategy is devised for adaptively select the best performing local search heuristic for each case, from a pool of general-purpose heuristics, so as to locally optimize the solutions during the evolution. We evaluated our propositions on a realistic multi-objective MSN search problem using trace-driven experiments with real mobility and social patterns. Extensive experimental studies reveal that the proposed method successfully learns the behaviour of individual local search heuristics during the evolution, adaptively follows the pattern of the best performing heuristics at different areas of the objective space and offers better performance in terms of both convergence and diversity than its competitors.The proposed Meta-Lamarckian based MOEA does not utilize any problem-specific heuristics, as most cases in the literature do, facilitating its applicability to other combinatorial MOO problems. To test its generalizability the proposed method is also evaluated on various test instances of the well-studied multi-objective Permutation Flow Shop Scheduling Problem.

A general system for heuristic minimization of convex functions over non-convex sets

We describe general heuristics to approximately solve a wide variety of problems with convex objective and decision variables from a non-convex set. The heuristics, which employ convex relaxations, convex restrictions, local neighbour search methods, and the alternating direction method of multipliers, require the solution of a modest number of convex problems, and are meant to apply to general problems, without much tuning. We describe an implementation of these methods in a package called NCVX, as an extension of CVXPY, a Python package for formulating and solving convex optimization problems. We study several examples of well known non-convex problems, and show that our general purpose heuristics are effective in finding approximate solutions to a wide variety of problems.

Nonlinear predictive control of a mobile robot: a solution using metaheuristcs

The basic features of model-based predictive control (MBPC) make it an interesting candidate for the control of mobile robots. However, fast solution procedures remain a challenge for nonlinear MBPC problems such as the one arising in mobile robot control. Metaheuristics are general purpose heuristics which have been successful in solving difficult optimization problems in a reasonable computation time. In this work, we present a comparison between the uses of three different heuristics, namely particle swarm optimization (PSO), ant colony optimization, and gravitational search algorithm for the solution of the nonlinear MBPC for a mobile robot tracking trajectory with dynamic obstacle avoidance. The computation times obtained show that PSO is a feasible alternative for real-time applications. The MBPC based on the PSO is applied to controlling a LEGO mobile robot with encouraged results.

A naturalistic decision making perspective on studying intuitive decision making

The Naturalistic Decision Making (NDM) community defines intuition as based on large numbers of patterns gained through experience, resulting in different forms of tacit knowledge. This view contrasts with Fast and Frugal Heuristics (FFH) researchers, who view intuition in terms of general purpose heuristics. The NDM view also differs from the Heuristics and Biases (HB) community, which sees intuitions as a source of bias and error. Seven suggestions are offered to assist the FFH and H&B communities in improving intuitive decision making and in conducting research that has greater potential for application.Rather than trying to help people analyze which option to choose, the NDM community recommends that intuitions be strengthened by providing a broader experience base that lets people build better tacit knowledge, such as perceptual skills and richer mental models, as a means of achieving better decisions.

Multi-Objective Predictive Control : A Solution Using Metaheuristics

The application of multi objective model predictive control approaches is significantly limited with computation time associated with optimization algorithms. Metaheuristics are general purpose heuristics that have been successfully used in solving difficult optimization problems in a reasonable computation time. In this work , we use and compare two multi objective metaheuristics, Multi-Objective Particle swarm Optimization, MOPSO, and Multi-Objective Gravitational Search Algorithm, MOGSA, to generate a set of approximately Pareto-optimal solutions in a single run. Two examples are studied, a nonlinear system consisting of two mobile robots tracking trajectories and avoiding obstacles and a linear multi variable system. The computation times and the quality of the solution in terms of the smoothness of the control signals and precision of tracking show that MOPSO can be an alternative for real time applications.

Assortment Optimization Under General Choice

We consider the key operational problem of optimizing the mix of offered products to maximize revenues when product prices are exogenously set and product demand follows a general discrete choice model. The key challenge in making this decision is the computational difficulty of finding the best subset, which often requires exhaustive search. Existing approaches address the challenge by either deriving efficient algorithms for specific parametric structures or studying the performance of general-purpose heuristics. The former approach results in algorithms that lack portability to other structures; whereas the latter approach has resulted in algorithms with poor performance. We study a portable and easy-to-implement local search heuristic. We show that it efficiently finds the global optimum for the multinomial logit (MNL) model and derive performance guarantees for general choice structures. Empirically, it is better than prevailing heuristics when no efficient algorithms exist, and it is within 0.02% of optimality otherwise.

Act first, think later: The presence and absence of inferential planning in problem solving

Planning is fundamental to successful problem solving, yet individuals sometimes fail to plan even one step ahead when it lies within their competence to do so. In this article, we report two experiments in which we explored variants of a ball-weighing puzzle, a problem that has only two steps, yet nonetheless yields performance consistent with a failure to plan. The results fit a computational model in which a solver's attempts are determined by two heuristics: maximization of the apparent progress made toward the problem goal and minimization of the problem space in which attempts are sought. The effectiveness of these heuristics was determined by lookahead, defined operationally as the number of steps evaluated in a planned move. Where move outcomes cannot be visualized but must be inferred, planning is constrained to the point where some individuals apply zero lookahead, which with n-ball problems yields seemingly irrational unequal weighs. Applying general-purpose heuristics with or without lookahead accounts for a range of rational and irrational phenomena found with insight and noninsight problems.

Algorithmic aspects of hardware/software partitioning

One of the most crucial steps in the design of embedded systems is hardware/software partitioning, that is, deciding which components of the system should be implemented in hardware and which ones in software. Most formulations of the hardware/software partitioning problem are NP-hard, so the majority of research efforts on hardware/software partitioning has focused on developing efficient heuristics.This article considers the combinatorial structure behind hardware/software partitioning. Two similar versions of the partitioning problem are defined, one of which turns out to be NP-hard, whereas the other one can be solved in polynomial time. This helps in understanding the real cause of complexity in hardware/software partitioning. Moreover, the polynomial-time algorithm serves as the basis for a highly efficient novel heuristic for the NP-hard version of the problem. Unlike general-purpose heuristics such as genetic algorithms or simulated annealing, this heuristic makes use of problem-specific knowledge, and can thus find high-quality solutions rapidly. Moreover, it has the unique characteristic that it also calculates lower bounds on the optimum solution . It is demonstrated on several benchmarks and also large random examples that the new algorithm clearly outperforms other heuristics that are generally applied to hardware/software partitioning.

An efficient genetic algorithm for job shop scheduling with tardiness objectives

We consider job shop scheduling problems with release and due-dates, as well as various tardiness objectives. To date, no efficient general-purpose heuristics have been developed for these problems. genetic algorithms can be applied almost directly, but come along with apparent weaknesses. We show that a heuristic reduction of the search space can help the algorithm to find better solutions in a shorter computation time. Two ways of reducing a search space are investigated by considering short-term decisions made at the machine level and by long-term decisions made at the shop floor level.

The footprint sorting problem.

Phylogenetic footprints are short pieces of noncoding DNA sequence in the vicinity of a gene that are conserved between evolutionary distant species. A seemingly simple problem is to sort footprints in their order along the genomes. It is complicated by the fact that not all footprints are collinear: they may cross each other. The problem thus becomes the identification of the crossing footprints, the sorting of the remaining collinear cliques, and finally the insertion of the noncollinear ones at "reasonable" positions. We show that solving the footprint sorting problem requires the solution of the "Minimum Weight Vertex Feedback Set Problem", which is known to be NP-complete and APX-hard. Nevertheless good approximations can be obtained for data sets of interest. The remaining steps of the sorting process are straightforward: computation of the transitive closure of an acyclic graph, linear extension of the resulting partial order, and finally sorting w.r.t. the linear extension. Alternatively, the footprint sorting problem can be rephrased as a combinatorial optimization problem for which approximate solutions can be obtained by means of general purpose heuristics. Footprint sortings obtained with different methods can be compared using a version of multiple sequence alignment that allows the identification of unambiguously ordered sublists. As an application we show that the rat has a slightly increased insertion/deletion rate in comparison to the mouse genome.

Accelerating Bounded Model Checking of Safety Properties

Bounded Model Checking based on SAT methods has recently been introduced as a complementary technique to BDD-based Symbolic Model Checking. The basic idea is to search for a counterexample in executions whose length is bounded by some integer k. The BMC problem can be efficiently reduced to a propositional satisfiability problem, and can therefore be solved by SAT methods rather than BDDs. SAT procedures are based on general-purpose heuristics that are designed for any propositional formula. We show how the unique characteristics of BMC invariant formulas (Gp) can be exploited for a variety of optimizations in the SAT checking procedure. Experiments with these optimizations on real designs prove their efficiency in many of the hard test cases, in comparison to both the standard SAT procedure and a BDD-based model checker.

The Attribute Based Hill Climber

In this paper we introduce the Attribute Based Hill Climber, a parameter-free algorithm that provides a concrete, stand-alone implementation of a little used technique from the Tabu Search literature known as “regional aspiration”. Results of applying the algorithm to two classical optimisation problems, the Travelling Salesman Problem and the Quadratic Assignment Problem, show it to be competitive with existing general purpose heuristics in these areas.

Meta optimization

Compiler writers have crafted many heuristics over the years to approximately solve NP-hard problems efficiently. Finding a heuristic that performs well on a broad range of applications is a tedious and difficult process. This paper introduces Meta Optimization, a methodology for automatically fine-tuning compiler heuristics. Meta Optimization uses machine-learning techniques to automatically search the space of compiler heuristics. Our techniques reduce compiler design complexity by relieving compiler writers of the tedium of heuristic tuning. Our machine-learning system uses an evolutionary algorithm to automatically find effective compiler heuristics. We present promising experimental results. In one mode of operation Meta Optimization creates application-specific heuristics which often result in impressive speedups. For hyperblock formation, one optimization we present in this paper, we obtain an average speedup of 23% (up to 73%) for the applications in our suite. Furthermore, by evolving a compiler's heuristic over several benchmarks, we can create effective, general-purpose heuristics. The best general-purpose heuristic our system found for hyperblock formation improved performance by an average of 25% on our training set, and 9% on a completely unrelated test set. We demonstrate the efficacy of our techniques on three different optimizations in this paper: hyperblock formation, register allocation, and data prefetching.

General Purpose Heuristics for Integer Programming—Part II

In spite of the many special purpose heuristics for specific classes of integer programming (IP) problems, there are few developments that focus on general purpose integer programming heuristics. This stems partly from the perception that general purpose methods are likely to be less effective than specialized procedures for specific problems, and partly from the perception that there is no unifying theoretical basis for creating general purpose heuristics. Still, there is a general acknowledgment that methods which are not limited to solving IP problems on a “class by class” basis, but which apply to a broader range of problems, have significant value. We provide a theoretical framework and associated explicit proposals for generating general purpose IP heuristics. Our development, makes use of cutting plane derivations that also give a natural basis for marrying heuristics with exact branch and cut methods for integer programming problems.