Stochastic Local Search Algorithms Research Articles

Improving two-mode algorithm via probabilistic selection for solving satisfiability problem

The satisfiability problem (SAT) is a critically important issue in multiple branches of computer science and artificial intelligence, with its relevance in industrial applications being of particular Significance CCAnr is the current leading stochastic local search (SLS) solver for tackling crafted satisfiable instances. It uses a two-mode strategy, greedy mode and diversification mode. In the present work, we employ a probabilistic selection approach to enhance CCAnr, leading to a new algorithm called ProbCCAnr. Experiments are carried out using the random SAT benchmarks and structured SAT benchmarks including instances encoded from mathematical problems and application problems. The experiments demonstrate that ProbCCAnr significantly improves the performance of state-of-the-art SLS algorithms including CCAnr and ProbSAT, among others. Moreover, ProbCCAnr shows better performance than state of the art complete solvers.

NuWLS: Improving Local Search for (Weighted) Partial MaxSAT by New Weighting Techniques

Maximum Satisfiability (MaxSAT) is a prototypical constraint optimization problem, and its generalized version is the (Weighted) Partial MaxSAT problem, denoted as (W)PMS, which deals with hard and soft clauses. Considerable progress has been made on stochastic local search (SLS) algorithms for solving (W)PMS, which mainly focus on clause weighting techniques. In this work, we identify two issues of existing clause weighting techniques for (W)PMS, and propose two ideas correspondingly. First, we observe that the initial values of soft clause weights have a big effect on the performance of the SLS solver for solving (W)PMS, and propose a weight initialization method. Second, we propose a new clause weighting scheme that for the first time employs different conditions for updating hard and soft clause weights. Based on these two ideas, we develop a new SLS solver for (W)PMS named NuWLS. Through extensive experiments, NuWLS performs much better than existing SLS solvers on all 6 benchmarks from the incomplete tracks of MaxSAT Evaluations (MSEs) 2019, 2020, and 2021. In terms of the number of winning instances, NuWLS outperforms state-of-the-art SAT-based incomplete solvers on all the 6 benchmarks. More encouragingly, a hybrid solver that combines NuWLS and an SAT-based solver won all four categories in the incomplete track of the MaxSAT Evaluation 2022.

Toward an Understanding of Long-tailed Runtimes of SLS Algorithms

The satisfiability problem (SAT) is one of the most famous problems in computer science. Traditionally, its NP-completeness has been used to argue that SAT is intractable. However, there have been tremendous practical advances in recent years that allow modern SAT solvers to solve instances with millions of variables and clauses. A particularly successful paradigm in this context is stochastic local search (SLS). In most cases, there are different ways of formulating the underlying SAT problem. While it is known that the precise formulation of the problem has a significant impact on the runtime of solvers, finding a helpful formulation is generally non-trivial. The recently introduced GapSAT solver [Lorenz and Wörz 2020] demonstrated a successful way to improve the performance of an SLS solver on average by learning additional information, which logically entails from the original problem. Still, there were also cases in which the performance slightly deteriorated. This justifies in-depth investigations into how learning logical implications affects runtimes for SLS algorithms. In this work, we propose a method for generating logically equivalent problem formulations, generalizing the ideas of GapSAT . This method allows a rigorous mathematical study of the effect on the runtime of SLS SAT solvers. Initially, we conduct empirical investigations. If the modification process is treated as random, then Johnson SB distributions provide a perfect characterization of the hardness. Since the observed Johnson SB distributions approach lognormal distributions, our analysis also suggests that the hardness is long-tailed. As a second contribution, we theoretically prove that restarts are useful for long-tailed distributions. This implies that incorporating additional restarts can further refine all algorithms employing above mentioned modification technique. Since the empirical studies compellingly suggest that the runtime distributions follow Johnson SB distributions, we also investigate this property on a theoretical basis. We succeed in proving that the runtimes for the special case of Schöning’s random walk algorithm [Schöning 2002] are approximately Johnson SB distributed.

Improving the performance of stochastic local search for maximum vertex weight clique problem using programming by optimization

The maximum vertex weight clique problem (MVWCP) is an important generalization of the maximum clique problem (MCP) that has a wide range of real-world applications. In situations where rigorous guarantees regarding the optimality of solutions are not required, MVWCP is usually solved using stochastic local search (SLS) algorithms, which also define the state of the art for solving this problem. However, there is no single SLS algorithm that gives the best performance across all classes of MVWCP instances, and it is challenging to effectively identify the most suitable algorithm for each class of MVWCP instances. In this work, we follow the paradigm of Programming by Optimization (PbO) to develop a new, flexible and highly parametric SLS framework for solving MVWCP, combining, for the first time, a broad range of effective heuristic mechanisms. By automatically configuring this PbO-MWC framework, we achieve substantial advances in the state of the art in solving MVWCP over a broad range of prominent benchmarks, including two derived from real-world applications in transplantation medicine (kidney exchange) and assessment of research excellence.

Disjunctive Rule Lists

In this study, we present an interpretable model, disjunctive rule list (DisRL) for regression. This research is motivated by the increasing need for model interpretability, especially in high-stakes decisions such as medicine, where decisions are made on or related to humans. DisRL is a generalized form of rule lists. A DisRL model consists of a list of disjunctive rules embedded in an if-else logic structure that stratifies the data space. Compared with traditional decision trees and other rule list models in the literature that stratify the feature space with single itemsets (an itemset is a conjunction of conditions), each disjunctive rule in DisRL uses a set of itemsets to collectively cover a subregion in the feature space. In addition, a DisRL model is constructed under a global objective that balances the predictive performance and model complexity. To train a DisRL model, we devise a hierarchical stochastic local search algorithm that exploits the properties of DisRL’s unique structure to improve search efficiency. The algorithm adopts the main structure of simulated annealing and customizes the proposing strategy for faster convergence. Meanwhile, the algorithm uses a prefix bound to locate a subset of the search area, effectively pruning the search space at each iteration. An ablation study shows the effectiveness of this strategy in pruning the search space. Experiments on public benchmark datasets demonstrate that DisRL outperforms baseline interpretable models, including decision trees and other rule-based regressors. History: Accepted by J. Paul Brooks, Area Editor for Applications in Biology, Medicine, & Healthcare. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplementary Information [ https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.1242 ] or is available from the IJOC GitHub software repository ( https://github.com/INFORMSJoC ) at [ http://dx.doi.org/10.5281/zenodo.6954927 ].

Improving probability selection based weights for satisfiability problems

Boolean Satisfiability problem (SAT) plays a prominent role in many domains of computer science and artificial intelligence due to its significant importance in both theory and applications. Algorithms for solving SAT problems can be categorized into two main classes: complete algorithms and incomplete algorithms (typically stochastic local search (SLS) algorithms). SLS algorithms are among the most effective for solving uniform random SAT problems, while hybrid algorithms achieved great breakthroughs for solving hard random SAT (HRS) problem recently. However, there is a lack of algorithms that can effectively solve both uniform random SAT and HRS problems. In this paper, a new SLS algorithm named SelectNTS is proposed aiming at solving both uniform random SAT and HRS problem effectively. SelectNTS is essentially an improved probability selection based local search algorithm, the core of which includes new clause and variable selection heuristics: a new clause weighting scheme and a biased random walk strategy are utilized to select a clause, while a new probability selection strategy with the variation of configuration checking strategy is used to select a variable. Extensive experimental results show that SelectNTS outperforms the state-of-the-art random SAT algorithms and hybrid algorithms in solving both uniform random SAT and HRS problems effectively.

Generalization and completeness of stochastic local search algorithms

We generalize Stochastic Local Search (SLS) heuristics into a unique formal model. This model has two key components: a common structure designed to be as large as possible and a parametric structure intended to be as small as possible. Each heuristic is obtained by instantiating the parametric part in a different way. Particular instances for Genetic Algorithms (GA), Ant Colony Optimization (ACO), and Particle Swarm Optimization (PSO) are presented. Then, we use our model to prove the Turing-completeness of SLS algorithms in general. The proof uses our framework to construct a GA able to simulate any Turing machine. This Turing-completeness implies that determining any non-trivial property concerning the relationship between the inputs and the computed outputs is undecidable for GA and, by extension, for the general set of SLS methods (although not necessarily for each particular method). Similar proofs are more informally presented for PSO and ACO.

Configuration Checking with Aspiration in Local Search for SAT

An interesting strategy called configuration checking (CC) was recently proposed to handle the cycling problem in local search for Minimum Vertex Cover. A natural question is whether this CC strategy also works for SAT. The direct application of CC did not result in stochastic local search (SLS) algorithms that can compete with the current best SLS algorithms for SAT. In this paper, we propose a new heuristic based on CC for SLS algorithms for SAT, which is called configuration checking with aspiration (CCA). It is used to develop a new SLS algorithm called Swcca. The experiments on random 3-SAT instances show that Swcca significantly outperforms Sparrow2011, the winner of the random satisfiable category of the SAT Competition 2011, which is considered to be the best local search solver for random 3-SAT instances. Moreover, the experiments on structured instances show that Swcca is competitive with Sattime, the best local search solver for the crafted benchmark in the SAT Competition 2011.

The Efficiency of Discrete Optimization Algorithm Portfolios

Introduction. Solving large-scale discrete optimization problems requires the processing of large-scale data in a reasonable time. Efficient solving is only possible by using multiprocessor computer systems. However, it is a daunting challenge to adapt existing optimization algorithms to get all the benefits of these parallel computing systems. The available computational resources are ineffective without efficient and scalable parallel methods. In this connection, the algorithm unions (portfolios and teams) play a crucial role in the parallel processing of discrete optimization problems. The purpose. The purpose of this paper is to research the efficiency of the algorithm portfolios by solving the weighted max-cut problem. The research is carried out in two stages using stochastic local search algorithms. Results. In this paper, we investigate homogeneous and non-homogeneous algorithm portfolios. We developed the homogeneous portfolios of two stochastic local optimization algorithms for the weighted max-cut problem, which has numerous applications. The results confirm the advantages of the proposed methods. Conclusions. Algorithm portfolios could be used to solve well-known discrete optimization problems of unprecedented scale and significantly improve their solving time. Further, we propose using communication between algorithms, namely teams and portfolios of algorithm teams. The algorithms in a team communicate with each other to boost overall performance. It is supposed that algorithm communication allows enhancing the best features of the developed algorithms and would improve the computational times and solution quality. The underlying algorithms should be able to utilize relevant data that is being communicated effectively to achieve any computational benefit from communication. Keywords: Discrete optimization, algorithm portfolios, computational experiment.

Stochastic Local Search for the Direct Aperture Optimisation Problem

Intensity modulated radiation therapy (IMRT) is a widely applied cancer treatment technique that aims at effectively deliver radiation to cancerous cells while sparing surrounding healthy tissues. To this end, the radiation coming from a linear accelerator is modulated using a physical device called multi-leaf collimator.Traditionally, a sequential approach is applied to generate treatment plans in IMRT. In this approach, the optimal intensities for each beam angle are computed (Fluence Map Optimisation, FMO), and then a sequencing problem is solved to obtain the collimator setup (aperture shapes) required to achieve the intensities computed for the FMO problem. The sequencing step is critical as the treatment plans computed to solve the FMO problem are not clinically acceptable. Unfortunately, treatment plans obtained by the sequencing step are severely impaired w.r.t. those obtained in the FMO step. One approach that aims to deal with the problem described above is the Direct Aperture Optimisation (DAO) approach. The DAO problem aims at simultaneously determining deliverable aperture shapes and a set of radiation intensities. This approach considers both physical and delivery time constraints, allowing to generate clinically acceptable treatment plans.In this paper, we propose a stochastic local search algorithm to solve the DAO problem. Our approach alternates the search between two neighbourhood definitions; the first focused on the fluence map (intensities) and the second on the aperture shapes. We analyse the algorithmic components, parameters, and overall results obtained by our algorithm on a set of clinical prostate cancer cases. We compare the obtained treatment plans to the ones obtained by the traditional two-step approach. Results show that our algorithm is able to find deliverable treatment plans without major impairments in treatment plans quality.

Stochastic Local Search for Solving Chance-Constrained Multi-Manned U-shaped Assembly Line Balancing Problem with Time and Space Constraints

The assembly line balancing problems have great importance in research and industry fields. They allow minimizing the learning aspects and guaranteeing a fixed number of products per day. This paper introduces a new problem that combines the multi-manned concept with the U-shaped lines with time and space constraints under uncertainty. The processing time of the tasks is considered as random variables with known means and variances. Therefore, chance-constraints appear in the cycle time constraints. In addition, each task has an associated area, where the assigned tasks per station are restricted by a total area. The proposed algorithm for solving the problem is a stochastic local search algorithm. The parameter levels of the proposed algorithm are optimized by the Taguchi method to cover the small, medium, and large-sized problems. Well-known benchmark problems have been adapted to cover the new model. The computational results showed the importance of the new problem and the efficiency of the proposed algorithm.

Low-Autocorrelation Random Number Generator Based on Adiabatic Quantum-Flux-Parametron Logic

We propose a true random number generator (TRNG) based on adiabatic quantum-flux-parametron (AQFP) logic. This AQFP TRNG corresponds to an AQFP buffer chain with the input current removed, which enables the generation of random bits via thermal fluctuation. To realize ultralow autocorrelation, several isolation inductors are inserted between adjacent buffers in the AQFP TRNG, thereby reducing the impact of the back action between buffers. We design and fabricate an AQFP TRNG using the AIST 10 kA/cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Nb high-speed standard process and evaluate the quality of the generated random bit streams. For a 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">17</sup> -bit random stream obtained from the AQFP TRNG at 100 kHz, the autocorrelation function is 5.00 × 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-3</sup> . Furthermore, we evaluate a 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">20</sup> -bit random stream from the AQFP TRNG using a statistical test suite provided by the National Institute of Standards and Technology. The bit stream passed 14 out of 15 tests, which suggests that the proposed AQFP TRNG can be used for applications such as stochastic computing and stochastic local search algorithms.

Emphasis on the flipping variable: Towards effective local search for hard random satisfiability

Uniform random satisfiability (URS) and hard random satisfiability (HRS) are two important benchmarks for algorithms that solve Boolean satisfiability problems, i.e., SAT solvers, especially for random SAT solvers. Recently, the stochastic local search (SLS) algorithms have made major breakthroughs in URS, resulting in several new state-of-the-art algorithms, e.g., Dimetheus, YalSAT, ProbSAT, and Score2SAT. However, compared to the great progress of SLS on URS, the performance of SLS on HRS lags far behind. In this paper, we propose a new SLS algorithm, named EPEFV for HRS, which employs the extended framework of ProbSAT, and adds a new heuristic method that emphasizes the role of flipping variable, called EFV. EFV focuses on the flipping variables and is based on three components: 1) A new clause weighting scheme focusing on the flipping variable, which is based on a new clause property called UnsatT. By applying this new weighting scheme and a biased random walk, we design a new clause selection mechanism. 2) Design a new scoring function named Uv by combining a novel variable property vUnsatT based on the flipping variable with the commonly used property score.3) A new tie-breaking strategy in the variable selection mechanism based on the new scoring function Uv. Extensive experimental results demonstrate that EPEFV can not only greatly outperforms the state-of-the-art SLS algorithms as well as complete solver competitors on HRS instances, but also can effectively solve URS instances with long clauses. On the contrary, the most advanced SLS solvers, however, can only effectively solve URS instances, while the most advanced complete solvers can only effectively solve HRS instances. At present, no solver can effectively solve both HRS and URS at the same time, which means that the EPEFV can be regarded as the state-of-the-art SLS solver for both HRS instances and URS instances with long clauses. Finally, further empirical analysis confirms the effectiveness of each mechanism underlying the EFV heuristic on HRS instances.

Stochastic Local Search Algorithms for Feature Selection: A Review

In today’s world, the data generated by many applications are increasing drastically, and finding an optimal subset of features from the data has become a crucial task. The main objective of this review is to analyze and comprehend different stochastic local search algorithms to find an optimal feature subset. Simulated annealing, tabu search, genetic programming, genetic algorithm, particle swarm optimization, artificial bee colony, grey wolf optimization, and bat algorithm, which have been used in feature selection, are discussed. This review also highlights the filter and wrapper approaches for feature selection. Furthermore, this review highlights the main components of stochastic local search algorithms, categorizes these algorithms in accordance with the type, and discusses the promising research directions for such algorithms in future research of feature selection.

Stochastic local search: a state-of-the-art review

The main objective of this paper is to provide a state-of-the-art review, analyze and discuss stochastic local search techniques used for solving hard combinatorial problems. It begins with a short introduction, motivation and some basic notation on combinatorial problems, search paradigms and other relevant features of searching techniques as needed for background. In the following a brief overview of the stochastic local search methods along with an analysis of the state-of-the-art stochastic local search algorithms is given. Finally, the last part of the paper present and discuss some of the most latest trends in application of stochastic local search algorithms in machine learning, data mining and some other areas of science and engineering. We conclude with a discussion on capabilities and limitations of stochastic local search algorithms.

Automatic design of hybrid stochastic local search algorithms for permutation flowshop problems with additional constraints

Automatic design of stochastic local search algorithms has been shown to be very effective in generating algorithms for the permutation flowshop problem for the most studied objectives including makespan, flowtime and total tardiness. The automatic design system uses a configuration tool to combine algorithmic components following a set of rules defined as a context-free grammar. In this paper we use the same system to tackle two of the most studied additional constraints for these objectives: sequence dependent setup times and no-idle constraint. Additional components have been added to adapt the system to the new problems while keeping intact the grammar structure and the experimental setup. The experiments show that the generated algorithms outperform the state of the art in each case.

A Stochastic Local Search Algorithm for the Partial Max-SAT Problem Based on Adaptive Tuning and Variable Depth Neighborhood Search

The Partial Max-SAT (PMSAT) problem is an optimization variant of the well-known Propositional Boolean Satisfiability (SAT) problem. It holds an important place in theory and practice, because a huge number of real-world problems, such as timetabling, planning, routing, bioinformatics, fault diagnosis, etc., could be encoded into it. Stochastic local search (SLS) methods can solve many real-world problems that often involve large-scale instances at reasonable computation costs while delivering good-quality solutions. In this work, we propose a novel SLS algorithm called adaptive variable depth SLS for PMSAT problem solving based on a dynamic local search framework. Our algorithm exploits two algorithmic components of an SLS method: parameter tuning and neighborhood search. Our first contribution is the design of an adaptive parameter tuner that searches for the best parameter setting for each instance by considering its features. The second contribution is a variable depth neighborhood search (VDS) algorithm adopted for PMSAT problem, which our empirical evaluation proves is a more efficient w.r.t. single neighborhood search. We conducted our experiments on the PMSAT benchmarks from MaxSAT Evaluation 2014 to 2019, including more than 3600 instances which have been encoded from a broad range of domains such as verification, optimization, graph theory, automated-reasoning, pseudo Boolean, etc. Our experimental evaluation results show that AVD-SLS solver, which is implemented based on our algorithm, outperforms state-of-the-art PMSAT SLS solvers in most benchmark classes, including random, crafted, and industrial instances. Furthermore, AVD-SLS reports remarkably better results on weighted benchmark, and shows competitive results with several well-known hybrid PMSAT solvers.

More efficient stochastic local search for satisfiability

Uniform random satisfiability (URS) and hard random satisfiability (HRS) are two significant generalizations of random satisfiability (RS). Recently, great breakthroughs have been made on stochastic local search (SLS) algorithms for uniform RS, resulting in several state-of-the-art algorithms, e.g., Dimetheus, YalSAT, ProbSAT and Score2SAT. However, compared to the great progress of SLS on URS, the performance of SLS on HRS lags far behind. In this paper, we propose two global clause weighting schemes and a new hybrid scoring function called SA based on a linear combination of a property score and property age, and then apply a second-level-biased random walk strategy based on two clause weighting schemes and SA to develop a new SLS solver called BRSAP. To evaluate the performance of BRSAP, we conduct extensive experiments to compare BRSAP with state-of-the-art SLS solvers and complete solvers on HRS instances and URS instances from SAT Competition 2017 and SAT Competition 2018 as well as 4100 generated satisfiable large HRS and URS ones. The experiments illustrate that BRSAP obviously outperforms its competitors, indicating the effectiveness of BRSAP. We also analyze the effectiveness of the underlying ideas in BRSAP.

Evaluating the impact of grammar complexity in automatic algorithm design

AbstractGrammar‐based automatic algorithm design has been shown to generate stochastic local search algorithms that compete with or outperform state‐of‐the‐art methods. In such systems, algorithms are divided in components and a grammar is used to describe how to properly combine the components to create a working algorithm. In our approach, the grammar is converted into parameters and an automatic parameter configuration tool is used to find the best configuration. This approach allows us to consider and hybridize different metaheuristic templates producing combinations never tested before, but this flexibility leads to a very large configuration space to explore. Is such complexity really needed to achieve state‐of‐the‐art performance? In this paper, we investigate this question by creating grammars that allow the hybridization of stochastic local search algorithms at most two, one, or zero times. We use these grammars to generate algorithms for the three most studied objectives of the permutation flowshop problem: makespan, total completion time, and total tardiness. The generated algorithms are compared using benchmark sets from the literature as well as a quantitative measure of algorithm complexity using a metric based on concept directed acyclic graphs. The experiments show that our system tends to generate hybridized algorithms only when they can provide a substantial performance improvement. On the contrary, when such algorithms do not improve performance, the system generates simpler algorithms regardless of the grammar used.

On conflict free DNA codes

DNA storage has emerged as an important area of research. The reliability of DNA storage system depends on designing the DNA strings (called DNA codes) that are sufficiently dissimilar. In this work, we introduce DNA codes that satisfy a special constraint. Each codeword of the DNA code has a specific property that any two consecutive sub-strings of the DNA codeword will not be the same (a generalization of homo-polymers constraint). This is in addition to the usual constraints such as Hamming, reverse, reverse-complement and $GC$-content. We believe that the new constraint will help further in reducing the errors during reading and writing data into the synthetic DNA strings. We also present a construction (based on a variant of stochastic local search algorithm) to calculate the size of the DNA codes with all the above constraints, which improves the lower bounds from the existing literature, for some specific cases. Moreover, a recursive isometric map between binary vectors and DNA strings is proposed. Using the map and the well known binary codes we obtain few classes of DNA codes with all the constraints including the property that the constructed DNA codewords are free from the hairpin-like secondary structures.