Hard Instances Research Articles

Sampling diverse near-optimal solutions via algorithmic quantum annealing.

Sampling a diverse set of high-quality solutions for hard optimization problems is of great practical relevance in many scientific disciplines and applications, such as artificial intelligence and operations research. One of the main open problems is the lack of ergodicity, or mode collapse, for typical stochastic solvers based on Monte Carlo techniques leading to poor generalization or lack of robustness to uncertainties. Currently, there is no universal metric to quantify such performance deficiencies across various solvers. Here, we introduce a new diversity measure for quantifying the number of independent approximate solutions for NP-hard optimization problems. Among others, it allows benchmarking solver performance by a required time-to-diversity (TTD), a generalization of often used time-to-solution (TTS). We illustrate this metric by comparing the sampling power of various quantum annealing strategies. In particular, we show that the inhomogeneous quantum annealing schedules can redistribute and suppress the emergence of topological defects by controlling space-time separated critical fronts, leading to an advantage over standard quantum annealing schedules with respect to both TTS and TTD for finding rare solutions. Using path-integral Monte Carlo simulations for up to 1600 qubits, we demonstrate that nonequilibrium driving of quantum fluctuations, guided by efficient approximate tensor network contractions, can significantly reduce the fraction of hard instances for random frustrated 2D spin glasses with local fields. Specifically, we observe that by creating a class of algorithmic quantum phase transitions, the diversity of solutions can be enhanced by up to 40% with the fraction of hard-to-sample instances reducing by more than 25%.

An exact constraint programming based procedure for the multi-manned assembly line balancing problem

Unlike the Simple Assembly Line Balancing Problem (SALBP), the Multi-manned Assembly Line Balancing Problem (MALBP) allows the assignment of multiple workers at the same station. This study proposes a novel Constraint Programming (CP) model to evaluate the satisfiability of a type-F MALBP. Given a cycle time limit, the number of workers and the number of stations are set as parameters in each iteration. We adopt an exact lexicographical procedure as the solution method for this multi-objective optimization problem, performing a lower-bound search to minimize the number of workers as the primary objective. Such an assumption is common for real-world applications, in which the total cost of a worker is usually several orders of magnitude higher than the cost of a station. When dealing with hard instances, we adopt the task–worker assignment solution for SALBP as input to an auxiliary Mixed-Integer Linear Programming (MILP) model, which provides an initial condition for the task scheduling portion of the CP model. Strong tasks’ scheduling bounds are also provided to tighten the search space, and mathematical similarities with SALBP are exploited to support our solution method. The integrated MILP-CP procedure yielded 126 optimal solutions from a 140-instance dataset, including nine new optimality proofs. Moreover, twelve additional instances were improved compared to the literature, enhancing the solution quality. Considering the best-found primal bounds, the maximum gap for the still-open instances is just 0.32%. These results demonstrated the effectiveness of the proposed procedure in solving MALBP when combined with solid bounds.

Fast One-to-Many Multicriteria Shortest Path Search

Shortest path problem has been successfully applied in numerous domains. Unfortunately, its complexity increases drastically when several objective criteria must be considered. Apart from the relatively slow classic search algorithms, attempts to accelerate multicriteria shortest path search are mostly represented by goal-directed one-to-one search methods and pruning heuristics. The one-to-many version of the problem is rarely addressed, though it arises in various scenarios, such as multi-stop planning and dynamic rerouting. This paper introduces a novel algorithm combination designed for fast one-to-many multicriteria shortest path search. A preprocessing algorithm excludes irrelevant vertices by building a smaller cover graph. A modified version of the multicriteria label-setting algorithm operates on the cover graph and employs a dimensionality reduction technique for swifter domination checks. While the method itself maintains solution optimality, it is able to additionally incorporate existing heuristics for further speedups. Additionally, its operation is not limited to bicriteria cases and requires no modifications to incorporate a higher number of criteria. The proposed algorithm was tested on multiple criteria combinations of varying correlation and compared to existing one-to-one shortest path search techniques. The results show the introduced approach provides a speedup of at least 3 times on simple criteria combinations and at least over 24 times on hard instances compared to standard multicriteria label-setting, while outperforming existing one-to-one algorithms in terms of scalability. Apart from the speedup provided, graph preprocessing also reduces memory requirements of queries by up to 13 times.

Probabilistic Generalization of Backdoor Trees with Application to SAT

The concept of Strong Backdoor Sets (SBS) for Constraint Satisfaction Problems is well known as one of the attempts to exploit structural peculiarities in hard instances. However, in practice, finding an SBS for a particular instance is often harder than solving it. Recently, a probabilistic weakened variant of the SBS was introduced: in the SBS, all subproblems must be polynomially solvable, whereas in the probabilistic SBS only a large fraction ρ of them should have this property. This new variant of backdoors called ρ-backdoors makes it possible to use the Monte Carlo method and metaheuristic optimization to find ρ-backdoors with ρ very close to 1, and relatively fast. Despite the fact that in a ρ-backdoor-based decomposition a portion of hard subproblems remain, in practice the narrowing of the search space often allows solving the problem faster with such a backdoor than without it. In this paper, we significantly improve on the concept of ρ-backdoors by extending this concept to backdoor trees: we introduce ρ-backdoor trees, show the interconnections between SBS, ρ-backdoors, and the corresponding backdoor trees, and establish some new theoretical properties of backdoor trees. In the experimental part of the paper, we show that moving from the metaheuristic search for ρ-backdoors to that of ρ-backdoor trees allows drastically reducing the time required to construct the required decompositions without compromising their quality.

Comparing the hardness of MAX 2-SAT problem instances for quantum and classical algorithms

We investigate whether small sized problem instances, which are commonly used in numerical simulations of quantum algorithms for benchmarking purposes, are a good representation of larger instances in terms of hardness. We approach this through a numerical analysis of the hardness of MAX 2-SAT problem instances for various continuous-time quantum algorithms and a comparable classical algorithm. Our results can be used to predict the viability of hybrid approaches that combine multiple algorithms in parallel to take advantage of the variation in the hardness of instances between the algorithms. We find that, while there are correlations in instance hardness between all of the algorithms considered, they appear weak enough that a hybrid strategy would likely be desirable in practice. Our results also show a widening range of hardness of randomly generated instances as the problem size is increased, which demonstrates both the difference in the distribution of hardness at small sizes and the value of a hybrid approach that can reduce the number of extremely hard instances. We identify specific weaknesses that can be overcome with hybrid techniques, such as the inability of these quantum algorithms to efficiently solve satisfiable instances (which is easy classically).

Diabatic quantum and classical annealing of the Sherrington-Kirkpatrick model

Quantum annealing is a contender to solve combinatorial optimization problems based on quantum dynamics. While significant efforts have been undertaken to investigate the quality of the solutions and the required run times, much less attention has been paid to understanding the dynamics of quantum annealing and the process leading to the solution during the sweep itself. In this comprehensive study, we investigate various aspects of the quantum annealing dynamics using different approaches. We perform quantum annealing, simulated quantum annealing, and classical annealing on several hundred instances of the Sherrington-Kirkpatrick model with intermediate system sizes up to 22 spins using numerical simulations. We observe qualitative differences between the quantum and classical methods, in particular at intermediate times, where a peak in the fidelity, also known as diabatic bump, appears for hard instances. Furthermore, we investigate the two-point correlation functions, which feature differences at intermediate times as well. At short times, however, the methods are similar again, which can be explained by relating the short-time expansion of quantum annealing to a high-temperature expansion, thus allowing one in principle to find the classical solution already at short times, albeit at prohibitive sampling cost.

Few-shot relation classification using clustering-based prototype modification

Few-shot relation classification is a natural language processing task that aims to enable models to recognize new relational categories of query instances by training on base classes with few labeled support instances. Many recently proposed prototypical-network-based approaches have exhibited excellent performance, some of which include additional information to further improve the model performance further. However, existing methods ignore the category information hidden in the query set and the hierarchical information that exists in relational hyponymy and fail to address the fundamental problem whereby the obtained prototype is always unreliable owing to the limited instances in the support set. In this study, we propose a novel prototypical-network-based model for the few-shot relation classification task to generate more precise relational prototypes by taking advantage of the category information hidden in the query set. In the model, we design a novel relational-similarity-based instance adaptive local loss that leverages relational hierarchical information to distinguish the hard instances belonging to similar relations effectively. Experiments on the FewRel dataset demonstrate that our approach achieves state-of-the-art performance.

Feasibility Jump: an LP-free Lagrangian MIP heuristic

We present Feasibility Jump (FJ), a primal heuristic for mixed-integer linear programs (MIP) using stochastic guided local search over a Lagrangian relaxation. The method is incomplete: it does not necessarily produce solutions to all feasible problems, the solutions it produces are not in general optimal, and it cannot detect infeasibility. It does, however, very quickly produce feasible solutions to many hard MIP problem instances. Starting from any variable assignment, Feasibility Jump repeatedly selects a variable and sets its value to minimize a weighted sum of constraint violations. These weights (which correspond to the Lagrangian multipliers) are adjusted for constraints that remain violated in local minima. Contrary to many other primal heuristics, Feasibility Jump does not require a solution of the continuous relaxation, which can be time-consuming for some problems. We compare FJ against FICO Xpress Solver 8.14 and we show that this heuristic is effective on a range of problems from the MIPLIB 2017 benchmark set, significantly improving the average time to find a first feasible solution. We also show that providing these quick solutions to Xpress produces a modest reduction in the average time to optimality in the same benchmark set. Our entry based on FJ to the MIP 2022 Computational Competition (which challenged participants to write LP-free MIP heuristics) won 1st place. Moreover, an implementation of Feasibility Jump now runs by default on FICO Xpress Solver 9.0, where similar results to the ones presented here could be observed.

A Hybrid Evolutionary Algorithm Using Two Solution Representations for Hybrid Flow-Shop Scheduling Problem.

As an extension of the classical flow-shop scheduling problem, the hybrid flow-shop scheduling problem (HFSP) widely exists in large-scale industrial production systems and has been considered to be challenging for its complexity and flexibility. Evolutionary algorithms based on encoding and heuristic decoding approaches are shown effective in solving the HFSP. However, frequently used encoding and decoding strategies can only search a limited area of the solution space, thus leading to unsatisfactory performance during the later period. In this article, a hybrid evolutionary algorithm (HEA) using two solution representations is proposed to solve the HFSP for makespan minimization. First, the proposed HEA searches the solution space by a permutation-based encoding representation and two heuristic decoding methods to find some promising areas. Afterward, a Tabu search (TS) procedure based on a disjunctive graph representation is introduced to expand the searching space for further optimization. Two classical neighborhood structures focusing on critical paths are extended to the problem-specific backward schedules to generate candidate solutions for the TS. The proposed HEA is tested on three public HFSP benchmark sets from the existing literature, including 567 instances in total, and is compared with some state-of-the-art algorithms. Extensive experimental results indicate that the proposed HEA performs much better than the other algorithms. Moreover, the proposed method finds new best solutions for 285 hard instances.

Beep-and-Sleep: Message and Energy Efficient Set Cover

This article considers message and energy-efficient distributed algorithms for the SetCover Problem. Given a ground set U of n elements and a set S of m subsets of U, we aim to find the minimal number of these subsets that contain all elements. In the default distributed setup of this problem, each set has a bidirected communication link with each element it contains. This results in a communication graph with n+m nodes and degree Δ. The value Δ denotes the maximal degree of the communication graph, i.e., the maximum of all subsets' sizes and the maximum number of sets an element is contained in.We present SetCover algorithm in the Beeping model that only relies on carrier-sensing. In each synchronous time step, a node can either listen to the channel or beep. A listening node learns if one or more of its neighbors beeped or if none of its neighbors beeped. In particular, it neither learns which neighbors beeped nor how many neighbors beeped exactly. Given this model, we present an algorithm that runs in O(k3) time and has an expected approximation ratio of O(Δ3/klog2⁡Δ). The value k∈[3,log⁡Δ] is a parameter that lets us trade runtime for approximation ratio similar to the celebrated algorithm by Kuhn and Wattenhofer [14]. Our next result is a O(k2)-time and O˜(Δ12+1k(n+m))-message algorithm (where O˜(⋅) hides polylogarithmic factors) with expected approximation ratio of O(Δ1klog⁡Δ) in the KT0-Congest model. In this variant of the well-known Congest model, time proceeds in synchronous rounds, and each node can send a distinct message of size O(log⁡(n+m)) to each of its neighbors. Further, each node has a unique identifier of size O(log⁡(n+m)). However, the crucial aspect of KT0-Congest is that the nodes do not know their neighbors' identifiers. Our algorithm is almost optimal concerning time and message complexity as we can show that there are hard instances that require Ω(Δ12−ϵm) messages for a constant approximation ratio.

Solving AC Optimal Power Flow with Discrete Decisions to Global Optimality

We present a solution framework for general alternating current optimal power flow (AC OPF) problems that include discrete decisions. The latter occur, for instance, in the context of the curtailment of renewables or the switching of power-generation units and transmission lines. Our approach delivers globally optimal solutions and is provably convergent. We model AC OPF problems with discrete decisions as mixed-integer nonlinear programs (MINLPs). The solution method starts from a known framework that uses piecewise linear relaxations. These relaxations are modeled as mixed-integer linear programs and adaptively refined until some termination criterion is fulfilled. In this work, we extend and complement this approach by problem-specific as well as very general algorithmic enhancements. In particular, these are mixed-integer second order cone programs as well as primal and dual cutting planes. For example, objective and no-good cuts help to compute good feasible solutions in which outer approximation constraints tighten the relaxations. We present extensive numerical results for various AC OPF problems in which discrete decisions play a major role. Even for hard instances with a large proportion of discrete decisions, the method is able to generate high-quality solutions efficiently. Furthermore, we compare our approach with state-of-the-art MINLP solvers. Our method outperforms all other algorithms. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms–Discrete. Funding: This research has been funded by the Federal Ministry of Education and Research of Germany [Grant 05M18WEB]. This research has been performed as part of the Energie Campus Nürnberg and is supported by funding of the Bavarian State Government. The authors thank the Deutsche Forschungsgemeinschaft for support within projects A05, B06, B07, and B10 of the Sonderforschungsbereich/Transregio 154 “Mathematical Modelling, Simulation and Optimization using the Example of Gas Networks.” This work has been supported by the Federal Ministry for Economic Affairs and Energy, Germany [Grant 03El1036A]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoc.2023.1270 .

Generating Random SAT Instances: Multiple Solutions could be Predefined and Deeply Hidden

The generation of SAT instances is an important issue in computer science, and it is useful for researchers to verify the effectiveness of SAT solvers. Addressing this issue could inspire researchers to propose new search strategies. SAT problems exist in various real-world applications, some of which have more than one solution. However, although several algorithms for generating random SAT instances have been proposed, few can be used to generate hard instances that have multiple predefined solutions. In this paper, we propose the KHidden-M algorithm to generate SAT instances with multiple predefined solutions that could be hard to solve by the local search strategy when the number of predefined solutions is small enough and the Hamming distance between them is not less than half of the solution length. Specifically, first, we generate an SAT instance that is satisfied by all of the predefined solutions. Next, if the generated SAT instance does not satisfy the hardness condition, then a strategy will be conducted to adjust clauses through multiple iterations to improve the hardness of the whole instance. We propose three strategies to generate the SAT instance in the first part. The first strategy is called the random strategy, which randomly generates clauses that are satisfied by all of the predefined solutions. The other two strategies are called the estimating strategy and greedy strategy, and using them, we attempt to generate an instance that directly satisfies or is closer to the hardness condition for the local search strategy. We employ two SAT solvers (i.e., WalkSAT and Kissat) to investigate the hardness of the SAT instances generated by our algorithm in the experiments. The experimental results show the effectiveness of the random, estimating and greedy strategies. Compared to the state-of-the-art algorithm for generating SAT instances with predefined solutions, namely, M-hidden, our algorithm could be more effective in generating hard SAT instances.

MIFNet: Multiple instances focused temporal action proposal generation

Temporal action proposal generation (TAPG) serves as a promising solution for video analysis. However, the performance of existing methods is still far from satisfactory for real-world applications. We attribute it to a crucial issue, i.e., hard multiple instances. In this paper, we investigate why this is the case. We discover that when processing multiple instances videos, mainstream approaches always recognize multiple instances as one instance due to boundary ambiguity or ignoring insignificant backgrounds between these instances. To address this problem, we propose a MultipleInstancesFocusedNetwork(MIFNet) that improves the quality of action proposals by considering boundary correlations and fusing multi-scale proposals. In particular, we first propose a pure boundary embedding module named Boundary Constraint Module (BCM) for suppressing the generation of hard negatives proposal by evaluating boundary correlation. The BCM introduces a boundary contrastive learning strategy that can pull the positive boundary pairs’ representation closer and push the negative pairs’ representation away. Then, a Proposal Blending Module (PBM) is proposed, which augments the proposal-level representation by modeling information among multi-scale proposals so that proposals can be complemented with local details as well as global information. The experimental results on the ActivityNet-v1.3 and THUMOS14 benchmarks demonstrate that MIFNet outperforms the state-of-the-arts.

Characterizing Tseitin-Formulas with Short Regular Resolution Refutations

Tseitin-formulas are systems of parity constraints whose structure is described by a graph. These formulas have been studied extensively in proof complexity as hard instances in many proof systems. In this paper, we prove that a class of unsatisfiable Tseitin-formulas of bounded degree has regular resolution refutations of polynomial length if and only if the treewidth of all underlying graphs G for that class is in O(log |V (G)|). It follows that unsatisfiable Tseitin-formulas with polynomial length of regular resolution refutations are completely determined by the treewidth of the underlying graphs when these graphs have bounded degree. To prove this, we show that any regular resolution refutation of an unsatisfiable Tseitin-formula with graph G of bounded degree has length 2Ω(tw(G))/|V (G)|, thus essentially matching the known 2O(tw(G))poly(|V (G)|) upper bound. Our proof first connects the length of regular resolution refutations of unsatisfiable Tseitin-formulas to the size of representations of satisfiable Tseitin-formulas in decomposable negation normal form (DNNF). Then we prove that for every graph G of bounded degree, every DNNF-representation of every satisfiable Tseitin-formula with graph G must have size 2Ω(tw(G)) which yields our lower bound for regular resolution.

Multiple Instance Complementary Detection and Difficulty Evaluation for Weakly Supervised Object Detection in Remote Sensing Images

Weakly supervised object detection (WSOD) in remote sensing images (RSIs) has attracted lots of attention because it solely employs image-level labels to drive the model training. Most of the WSOD methods incline to mine salient object as positive instance, and the less salient objects are considered as negative instances, which will cause the problem of missing instances. In addition, the quantity of hard and easy instances is usually imbalanced, consequently, the cumulative loss of a large amount of easy instances dominates the training loss, which limits the upper bound of WSOD performance. To handle the first problem, a complementary detection network (CDN) is proposed, which consists of a complementary multiple instance detection network (CMIDN) and a complementary feature learning (CFL) module. The CDN can capture robust complementary information from two basic multiple instance detection networks (MIDNs) and mine more object instances. To handle the second problem, an instance difficulty evaluation metric named instance difficulty score (IDS) is proposed, which is employed as the weight of each instance in the training loss. Consequently, the hard instances will be assigned larger weights according to the IDS, which can improve the upper bound of WSOD performance. The ablation experiments demonstrate that our method significantly increases the baseline method by large margins, i.e. 23.6% (10.2%) mAP and 32.4% (13.1%) CorLoc gains on the NWPU VHR-10.v2 (DIOR) dataset. Our method obtains 58.1% (26.7%) mAP and 72.4% (47.9%) CorLoc on the NWPU VHR-10.v2 (DIOR) dataset, which achieves better performance compared with seven advanced WSOD methods.

On the computational hardness of the code equivalence problem in cryptography

<p style='text-indent:20px;'>Code equivalence is a well-known concept in coding theory. Recently, literature saw an increased interest in this notion, due to the introduction of protocols based on the hardness of finding the equivalence between two linear codes. In this paper, we analyze the security of code equivalence, with a special focus on the hardest instances, in the interest of cryptographic usage. Our work stems from a thorough review of existing literature, identifies the various types of solvers for the problem, and provides a precise complexity analysis, where previously absent. Furthermore, we are able to improve on the state of the art, providing more efficient algorithm variations, for which we include numerical simulation data. In the end, the goal of this paper is to provide a complete, single point of access, which can be used as a tool for designing schemes that rely on the code equivalence problem.</p>

Online Progressive Instance-Balanced Sampling for Weakly Supervised Vibration Damper Detection

The detection of vibration dampers is important for power systems. Deep learning-based damper detection needs massive annotations, which are labour-intensive and time-consuming. Therefore, weakly supervised object detection is considered. Part domination is the main problem in weakly supervised vibration damper detection. Current weakly supervised object detection methods neglect that overwhelming negative instances exist in each image during the training phase, which would mislead the training and make detection results stuck in the most discriminative parts of objects. To tackle this problem, an online progressive instance-balanced sampling (OPIS) algorithm based on hard sampling and soft sampling is proposed in this paper. The algorithm includes two modules: a progressive instance balance (PIB) module and a progressive instance reweighting (PIR) module. The PIB module, combining random sampling and IoU-balanced sampling, progressively mines hard negative instances while balancing positive instances and negative instances. The PIR module further utilizes classifier scores and IoUs of adjacent refinements to reweight the weights of positive instances to make the network focus on positive instances. Extensive experimental results on the vibration damper, PASCAL VOC 2007, and PASCAL VOC 2012 datasets demonstrate that the proposed method can significantly improve the baseline, which is also comparable to many existing methods. In addition, compared to the baseline, the proposed method requires no extra network parameters, and the supplementary training overheads are small.

Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting.

While Graph Neural Networks (GNNs) have achieved remarkable results in a variety of applications, recent studies exposed important shortcomings in their ability to capture the structure of the underlying graph. It has been shown that the expressive power of standard GNNs is bounded by the Weisfeiler-Leman (WL) graph isomorphism test, from which they inherit proven limitations such as the inability to detect and count graph substructures. On the other hand, there is significant empirical evidence, e.g. in network science and bioinformatics, that substructures are often intimately related to downstream tasks. To this end, we propose "Graph Substructure Networks" (GSN), a topologically-aware message passing scheme based on substructure encoding. We theoretically analyse the expressive power of our architecture, showing that it is strictly more expressive than the WL test, and provide sufficient conditions for universality. Importantly, we do not attempt to adhere to the WL hierarchy; this allows us to retain multiple attractive properties of standard GNNs such as locality and linear network complexity, while being able to disambiguate even hard instances of graph isomorphism. We perform an extensive experimental evaluation on graph classification and regression tasks and obtain state-of-the-art results in diverse real-world settings including molecular graphs and social networks.

Compression for Multi-Arm Bandits

The multi-armed bandit (MAB) problem is one of the most well-known active learning frameworks. The aim is to select the best among a set of actions by sequentially observing rewards that come from an unknown distribution. Recently, a number of distributed bandit applications have become popular over wireless networks, where agents geographically separated from a learner collect and communicate the observed rewards. In this paper we propose a compression scheme, that compresses the rewards collected by the distributed agents. By providing nearly matching upper and lower bounds, we tightly characterize the number of bits needed per reward for the learner to accurately learn without suffering additional regret. In particular, we establish a generic reward quantization algorithm, that can be applied on top of any (no-regret) MAB algorithm to form a new communication-efficient counterpart. requires only a few (converging to as low as 3 bits as the number of iterations increases) bits to be sent per reward while preserving the same regret bound as uncompressed rewards. Our lower bound is established via constructing hard instances from a subgaussian distribution. Our theory is further corroborated by numerical experiments.

Generating hard satisfiable instances by planting into random constraint satisfaction problem model with growing constraint scope length

We propose generating hard satisfiable constraint satisfaction problem (CSP) instances by planting into the k-CSP model, which is a random CSP model with growing constraint scope length. The phase diagram of the planted k-CSP model is drawn in this paper, which has no differences from that of the pure random k-CSP model, except that the planted k-CSP model has a planted cluster while the pure random k-CSP model does not. The planted cluster, which is composed of the planted solution and solutions around it, is found to be very small. This phase diagram guarantees that the planted k-CSP instances are as hard as the pure random k-CSP instances before the satisfiability (by configurations outside the planted cluster) transition occurs, and hard satisfiable instances appear near the satisfiable transition point. Experiments on instances with 102 variables support that the model can generate hard satisfiable instances.