Submodular Minimization Research Articles

Solving Strong-Substitutes Product-Mix Auctions

This paper develops algorithms to solve strong-substitutes product-mix auctions: it finds competitive equilibrium prices and quantities for agents who use this auction’s bidding language to truthfully express their strong-substitutes preferences over an arbitrary number of goods, each of which is available in multiple discrete units. Our use of the bidding language and the information it provides contrasts with existing algorithms that rely on access to a valuation or demand oracle. We compute market-clearing prices using algorithms that apply existing submodular minimization methods. Allocating the supply among the bidders at these prices then requires solving a novel constrained matching problem. Our algorithm iteratively simplifies the allocation problem, perturbing bids and prices in a way that resolves tie-breaking choices created by bids that can be accepted on more than one good. We provide practical running time bounds on both price finding and allocation and illustrate experimentally that our allocation mechanism is practical. Funding: E. Baldwin and P. Klemperer were supported by the Economic and Social Research Council [Grant ES/L003058/1]. P. W. Goldberg and E. Lock were supported by a JP Morgan faculty fellowship during the work on the final version of the paper. Supplemental Material: The online companion is available at https://doi.org/10.1287/moor.2019.0248 .

Strong valid inequalities for a class of concave submodular minimization problems under cardinality constraints

We study the polyhedral convex hull structure of a mixed-integer set which arises in a class of cardinality-constrained concave submodular minimization problems. This class of problems has an objective function in the form of $$f(a^\top x)$$ , where f is a univariate concave function, a is a non-negative vector, and x is a binary vector of appropriate dimension. Such minimization problems frequently appear in applications that involve risk-aversion or economies of scale. We propose three classes of strong valid linear inequalities for this convex hull and specify their facet conditions when a has two distinct values. We show how to use these inequalities to obtain valid inequalities for general a that contains multiple values. We further provide a complete linear convex hull description for this mixed-integer set when a contains two distinct values and the cardinality constraint upper bound is two. Our computational experiments on the mean-risk optimization problem demonstrate the effectiveness of the proposed inequalities in a branch-and-cut framework.

Preference swaps for the stable matching problem

An instance I of the Stable Matching Problem (SMP) is given by a bipartite graph with a preference list of neighbors for every vertex. A swap in I is the exchange of two consecutive vertices in a preference list. A swap can be viewed as a smallest perturbation of I. Boehmer et al. (2021) designed a polynomial-time algorithm for finding the minimum number of swaps required to turn a given maximal matching into a stable matching. We generalize this result to the many-to-many version of SMP. We do so first by introducing a new representation of SMP as an extended bipartite graph and subsequently by reducing the problem to submodular minimization. It is a natural problem to establish the computational complexity of deciding whether at most k swaps are enough to turn I into an instance where one of the maximum matchings is stable. Using a hardness result of Gupta et al. (2020), we prove that this problem is NP-hard and, moreover, this problem parameterised by k is W[1]-hard. We also obtain a lower bound on the running time for solving the problem using the Exponential Time Hypothesis.

Constraint Satisfaction Problems with Global Modular Constraints: Algorithms and Hardness via Polynomial Representations

We study the complexity of Boolean constraint satisfaction problems (CSPs) when the assignment must have Hamming weight in some congruence class modulo $M$, for various choices of the modulus $M$. Due to the known classification of tractable Boolean CSPs, this mainly reduces to the study of three cases: 2-SAT, HORN-SAT, and LIN-2 (linear equations mod 2). We classify the moduli $M$ for which these respective problems are polynomial time solvable, and when they are not (assuming the exponential time hypothesis). Our study reveals that this modular constraint lends a surprising richness to these classic, well-studied problems, with interesting broader connections to complexity theory and coding theory. The HORN-SAT case is connected to the covering complexity of polynomials representing the NAND function mod $M$. The LIN-2 case is tied to the sparsity of polynomials representing the OR function mod $M$, which in turn has connections to modular weight distribution properties of linear codes and locally decodable codes. In both cases, the analysis of our algorithm as well as the hardness reduction rely on these polynomial representations, highlighting an interesting algebraic common ground between hard cases for our algorithms and the gadgets which show hardness. These new complexity measures of polynomial representations merit further study. The inspiration for our study comes from a recent work by Nägele, Sudakov, and Zenklusen on submodular minimization with a global congruence constraint. Our algorithm for HORN-SAT has strong similarities to their algorithm, and in particular identical kinds of set systems arise in both cases. Our connection to polynomial representations leads to a simpler analysis of such set systems and also sheds light on (but does not resolve) the complexity of submodular minimization with a congruency requirement modulo a composite $M$.

Censoring outliers in elliptically symmetric multivariate distribution

In this paper, we discuss the problem of censoring outliers in a dataset with a class of complex elliptically symmetric (CES) distribution, which has various engineering applications. To this end, the regularized cost function is formulated by the maximum likelihood (ML) setting. Since the ML estimation of the free outlier subset requires solving a combinatorial problem, an efficient and low complexity algorithm is proposed that guarantees the convergence to a stationary point. Two independent and dependent penalty terms to the selected subset are considered. The conventional generalized inner product (GIP) and the submodular minimization approaches are employed in the derivation of the proposed algorithm in the independent and dependent cases, respectively. The dependent case improves performance by slightly increasing complexity. As a special case for Gaussian assumption and using a non-regularized cost function, the proposed algorithm reduces to the fast minimum covariance determinant (FMCD) algorithm, which has impressive efficiency and received considerable attention in statistics and computer science. Hence, the proposed algorithm extends/generalizes the FMCD to be applicable to the non-Gaussian distributions. The simulation results highlight that the proposed algorithms exhibit satisfactory performance in CES environments.

Semi-Supervised Learning With Label Proportion

The scarcity of labels is common and great challenge in traditional supervised learning. Semi-supervised learning (SSL) leverages unlabeled samples to alleviate the absence of label information. Similar with annotation, label proportion is another type of prior information and plays a significant role in classification tasks. Compared with the acquisition of labels, label proportion can be obtained more easily. For example, only a small number of patients have been diagnosed with or not with cancers in hospital database, while the proportion with cancer can be generally estimated by historical records. How to incorporate such prior information of label proportion is crucial but rarely studied in literature. Traditional SSL methods often ignore this prior information and will lead to performance degradation inevitably. To solve this problem, we propose a novel SSL with Label Proportion (SSLLP). Our approach encourages to preserve label consistency and label proportion by imposing the cardinality bound constraints. Our formulated problem equals to a mixed-integer constrained submodular minimization and it is difficult to be solved directly. Therefore, we transformed the original problem into a convex one by Lov <inline-formula><tex-math notation="LaTeX">$\acute{\text{a}}$</tex-math></inline-formula> sz extension and designed an efficient solving algorithm. Extensive experimental results present the improved performance of our method over several state-of-the-art methods.

Distributed Optimization for Robot Networks: From Real-Time Convex Optimization to Game-Theoretic Self-Organization

Recent advances in sensing, communication, and computing technologies have enabled the use of multirobot systems for practical applications such as surveillance, area mapping, and search and rescue. For such systems, a major challenge is to design decision rules that are real-time-implementable, require local information only, and guarantee some desired global performance. Distributed optimization provides a framework for designing such local decision-making rules for multirobot systems. In this article, we present a collection of selected results for distributed optimization for robot networks. We will focus on two special classes of problems: 1) real-time path planning for multirobot systems and 2) self-organization in multirobot systems using game-theoretic approaches. For multirobot path planning, we will present some recent approaches that are based on approximately solving distributed optimization problems over continuous and discrete domains of actions. The main idea underlying these approaches is that a variety of path planning problems can be formulated as convex optimization and submodular minimization problems over continuous and discrete action spaces, respectively. To generate local update rules that are efficiently implementable in real time, these approaches rely on approximate solutions to the global problems that can still guarantee some level of desired global performance. For game-theoretic self-organization, we will present a sampling of results for area coverage and real-time target assignment. In these results, the problems are formulated as games, and online updating rules are designed to enable teams of robots to achieve the collective objective in a distributed manner.

Distributed Submodular Minimization via Block-Wise Updates and Communications

In this paper we deal with a network of computing agents with local processing and neighboring communication capabilities that aim at solving (without any central unit) a submodular optimization problem. The cost function is the sum of many local submodular functions and each agent in the network has access to one function in the sum only. In this distributed set-up, in order to preserve their own privacy, agents communicate with neighbors but do not share their local cost functions. We propose a distributed algorithm in which agents resort to the Lovàsz extension of their local submodular functions and perform local updates and communications in terms of single blocks of the entire optimization variable. Updates are performed by means of a greedy algorithm which is run only until the selected block is computed, thus resulting in a reduced computational burden. The proposed algorithm is shown to converge in expected value to the optimal cost of the problem, and an approximate solution to the submodular problem is retrieved by a thresholding operation. As an application, we consider a distributed image segmentation problem in which each agent has access only to a portion of the entire image. While agents cannot segment the entire image on their own, they correctly complete the task by cooperating through the proposed distributed algorithm.

Quantum and classical algorithms for approximate submodular function minimization

Submodular functions are set functions mapping every subset of some ground set of size n into the real numbers and satisfying the diminishing returns property. Submodular minimization is an important field in discrete optimization theory due to its relevance for various branches of mathematics, computer science and economics. The currently fastest strongly polynomial algorithm for exact minimization~\cite{LSW15} runs in time \so{n^3 \cdot \eo + n^4} where \eo denotes the cost to evaluate the function on any set. For functions with range [-1,1], the best \eps-additive approximation algorithm~\cite{CLSW17} runs in time \so{n^{5/3}/\eps^{2} \cdot \eo}. In this paper we present a classical and a quantum algorithm for approximate submodular minimization. Our classical result improves on the algorithm of \cite{CLSW17} and runs in time \so{n^{3/2}/\eps^2 \cdot \eo}. Our quantum algorithm is, up to our knowledge, the first attempt to use quantum computing for submodular optimization. The algorithm runs in time \so{n^{5/4}/\eps^{5/2} \cdot \log(1/\eps) \cdot \eo}. The main ingredient of the quantum result is a new method for sampling with high probability T independent elements from any discrete probability distribution of support size n in time \bo{\sqrt{Tn}}. Previous quantum algorithms for this problem were of complexity \bo{T\sqrt{n}}.

Submodular Minimization Under Congruency Constraints

Submodular function minimization (SFM) is a fundamental and efficiently solvable problem in combinatorial optimization with a multitude of applications in various fields. Surprisingly, there is only very little known about constraint types under which SFM remains efficiently solvable. The arguably most relevant non-trivial constraint class for which polynomial SFM algorithms are known are parity constraints, i.e., optimizing only over sets of odd (or even) cardinality. Parity constraints capture classical combinatorial optimization problems like the odd-cut problem, and they are a key tool in a recent technique to efficiently solve integer programs with a constraint matrix whose subdeterminants are bounded by two in absolute value. We show that efficient SFM is possible even for a significantly larger class than parity constraints, by introducing a new approach that combines techniques from Combinatorial Optimization, Combinatorics, and Number Theory. In particular, we can show that effi- cient SFM is possible over all sets (of any given lattice) of cardinality r mod m, as long as m is a constant prime power. This covers generalizations of the odd-cut problem with open complexity status, and has interesting links to integer programming with bounded subdeterminants. To obtain our results, we establish a connection between the correctness of a natural algorithm, and the nonexistence of set systems with specific combinatorial properties. We introduce a general technique to disprove the existence of such set systems, which allows for obtaining extensions of our results beyond the above-mentioned setting. These extensions settle two open questions raised by Geelen and Kapadia [9] in the context of computing the girth and cogirth of certain types of binary matroids.

Distributed Submodular Minimization and Motion Planning Over Discrete State Space

We develop a framework for the distributed minimization of submodular functions. Submodular functions are a discrete analog of convex functions and are extensively used in large-scale combinatorial optimization problems. While there has been a significant interest in the distributed formulations of convex optimization problems, distributed minimization of submodular functions has received relatively little research attention. Our framework relies on an equivalent convex reformulation of a submodular minimization problem, which is efficiently computable. We then use this relaxation to exploit methods for the distributed optimization of convex functions. The proposed framework is applicable to submodular set functions as well as to a wider class of submodular functions defined over certain lattices. We also propose an approach for solving distributed motion-planning problems in discrete state space based on submodular function minimization. We establish through a challenging setup of the capture the flag game that submodular functions over lattices can be used to design artificial potential fields for multiagent systems with discrete inputs. These potential fields are designed such that their minima correspond to desired behaviors, that is, agents are attracted toward their goals and are repulsed from obstacles and from each other for collision avoidance. Finally, we demonstrate that the proposed distributed framework can be employed effectively for generating feasible trajectories in such motion coordination problems.

Robust Budget Allocation Via Continuous Submodular Functions

The optimal allocation of resources for maximizing influence, spread of information or coverage, has gained attention in the past years, in particular in machine learning and data mining. But in applications, the parameters of the problem are rarely known exactly, and using wrong parameters can lead to undesirable outcomes. We hence revisit a continuous version of the Budget Allocation or Bipartite Influence Maximization problem introduced by Alon et al. (in: WWW’12 - Proceedings of the 21st Annual Conference on World Wide, ACM, New York, 2012) from a robust optimization perspective, where an adversary may choose the least favorable parameters within a confidence set. The resulting problem is a nonconvex–concave saddle point problem (or game). We show that this nonconvex problem can be solved exactly by leveraging connections to continuous submodular functions, and by solving a constrained submodular minimization problem. Although constrained submodular minimization is hard in general, here, we establish conditions under which such a problem can be solved to arbitrary precision $$\varepsilon $$ .

Strong formulations for quadratic optimization with M-matrices and indicator variables

We study quadratic optimization with indicator variables and an M-matrix, i.e., a PSD matrix with non-positive off-diagonal entries, which arises directly in image segmentation and portfolio optimization with transaction costs, as well as a substructure of general quadratic optimization problems. We prove, under mild assumptions, that the minimization problem is solvable in polynomial time by showing its equivalence to a submodular minimization problem. To strengthen the formulation, we decompose the quadratic function into a sum of simple quadratic functions with at most two indicator variables each, and provide the convex-hull descriptions of these sets. We also describe strong conic quadratic valid inequalities. Preliminary computational experiments indicate that the proposed inequalities can substantially improve the strength of the continuous relaxations with respect to the standard perspective reformulation.

Complexity and approximations for submodular minimization problems on two variables per inequality constraints

We demonstrate here that submodular minimization (SM) problems subject to constraints containing up to two variables per inequality, SM2, are 2-approximable in polynomial time and a better approximation factor cannot be achieved in polynomial time unless NP=P. The 2-approximation holds when either the constraints have the round-up property (the rounding up of a feasible fractional solution is feasible) or, if the constraints do not have this property, for monotone submodular functions. The submodular minimization or supermodular maximization on constraints where the coefficients of the two variables in each constraint are of opposite signs (monotone constraints) is solvable in polynomial time. The 2-approximability and the polynomial time solvability for monotone constraints hold also for multi-sets that contain elements with integer multiplicity greater than 1, except that the running time is then pseudo-polynomial in that it depends on the range of the variables. This complexity cannot be improved unless NP=P.Our results indicate that SM2 problems are not much harder than the respective linear integer problems on two variables per constraint: For monotone constraints both problems are polynomial time solvable, and for non-monotone NP-hard problems, both problems have 2-approximation algorithms. For SM2 problems the factor 2 approximation is provably best possible, whereas for the respective linear integer problems it has not been established that the factor 2 is best possible, but this has been conjectured. On the other hand, for SM2 problems where the two variables constraints’ coefficients form a totally unimodular constraint matrix, the linear integer optimization problem is solved in polynomial time, whereas the submodular optimization is proved here to be NP-hard.The submodular minimization NP-hard problems for which our general purpose 2-approximation algorithm applies include submodular-vertex cover, submodular-2SAT, submodular-min satisfiability, submodular-edge deletion for clique, submodular-node deletion for biclique and others.

SVitchboard-II and FiSVer-I: Crafting high quality and low complexity conversational english speech corpora using submodular function optimization

We introduce a set of benchmark corpora of conversational English speech derived from the Switchboard-I and Fisher datasets. Traditional automatic speech recognition (ASR) research requires considerable computational resources and has slow experimental turnaround times. Our goal is to introduce these new datasets to researchers in the ASR and machine learning communities in order to facilitate the development of novel speech recognition techniques on smaller but still acoustically rich, diverse, and hence interesting corpora. We select these corpora to maximize an acoustic quality criterion while limiting the vocabulary size (from 10 words up to 10,000 words), where both “acoustic quality” and vocabulary size are adeptly measured via various submodular functions. We also survey numerous submodular functions that could be useful to measure both “acoustic quality” and “corpus complexity” and offer guidelines on when and why a scientist may wish use to one vs. another. The corpora selection process itself is naturally performed using various state-of-the-art submodular function optimization procedures, including submodular level-set constrained submodular optimization (SCSC/SCSK), difference-of-submodular (DS) optimization, and unconstrained submodular minimization (SFM), all of which are fully defined herein. While the focus of this paper is on the resultant speech corpora, and the survey of possible objectives, a consequence of the paper is a thorough empirical comparison of the relative merits of these modern submodular optimization procedures. We provide baseline word recognition results on all of the resultant speech corpora for both Gaussian mixture model (GMM) and deep neural network (DNN)-based systems, and we have released all of the corpora definitions and Kaldi training recipes for free in the public domain.

Constrained Submodular Minimization for Missing Labels and Class Imbalance in Multi-label Learning

In multi-label learning, there are two main challenges: missing labels and class imbalance (CIB). The former assumes that only a partial set of labels are provided for each training instance while other labels are missing. CIB is observed from two perspectives: first, the number of negative labels of each instance is much larger than its positive labels; second, the rate of positive instances (i.e. the number of positive instances divided by the total number of instances) of different classes are significantly different. Both missing labels and CIB lead to significant performance degradation. In this work, we propose a new method to handle these two challenges simultaneously. We formulate the problem as a constrained submodular minimization that is composed of a submodular objective function that encourages label consistency and smoothness, as well as, class cardinality bound constraints to handle class imbalance. We further present a convex approximation based on the Lovasz extension of submodular functions, leading to a linear program, which can be efficiently solved by the alternative direction method of multipliers (ADMM). Experimental results on several benchmark datasets demonstrate the improved performance of our method over several state-of-the-art methods.

Polyhedral results for a class of cardinality constrained submodular minimization problems

Motivated by concave cost combinatorial optimization problems, we study the following mixed integer nonlinear set: P={(w,x)∈R×{0,1}n:w≥f(a′x),e′x≤k} where f:R→R is a concave function, n and k are positive integers, a∈Rn is a nonnegative vector, e∈Rn is a vector of ones, and x′y denotes the scalar product of vectors x and y of same dimension. A standard linearization approach for P is to exploit the fact that f(a′x) is submodular with respect to the binary vector x. We extend this approach to take the cardinality constraint e′x≤k into account and provide a full description of the convex hull of P when the vector a has identical components. We also develop a family of facet-defining inequalities when the vector a has nonidentical components. Computational results using the proposed inequalities in a branch-and-cut framework to solve mean-risk knapsack problems show significant decrease in both time and the number of nodes over standard methods.

Activity preserving graph simplification

We study the problem of simplifying a given directed graph by keeping a small subset of its arcs. Our goal is to maintain the connectivity required to explain a set of observed traces of information propagation across the graph. Unlike previous work, we do not make any assumption about an underlying model of information propagation. Instead, we approach the task as a combinatorial problem. We prove that the resulting optimization problem is \(\mathbf{NP}\)-hard. We show that a standard greedy algorithm performs very well in practice, even though it does not have theoretical guarantees. Additionally, if the activity traces have a tree structure, we show that the objective function is supermodular, and experimentally verify that the approach for size-constrained submodular minimization recently proposed by Nagano et al. (28th International Conference on Machine Learning, 2011) produces very good results. Moreover, when applied to the task of reconstructing an unobserved graph, our methods perform comparably to a state-of-the-art algorithm devised specifically for this task.

The complexity of conservative valued CSPs

We study the complexity of valued constraint satisfaction problems (VCSPs) parametrized by aconstraint language, a fixed set of cost functions over a finite domain. An instance of the problem is specified by a sum of cost functions from the language and the goal is to minimize the sum. Under the unique games conjecture, the approximability of finite-valued VCSPs is well understood, see Raghavendra [2008]. However, there is no characterization of finite-valued VCSPs, let alone general-valued VCSPs, that can be solved exactly in polynomial time, thus giving insights from a combinatorial optimization perspective.We consider the case of languages containing all possible unary cost functions. In the case of languages consisting of only {0,∞}-valued cost functions (i.e., relations), such languages have been calledconservativeand studied by Bulatov [2003, 2011] and recently by Barto [2011]. Since we study valued languages, we call a languageconservativeif it contains all finite-valued unary cost functions. The computational complexity of conservative valued languages has been studied by Cohen et al. [2006] for languages over Boolean domains, by Deineko et al. [2008] for {0,1}-valued languages (a.k.a Max-CSP), and by Takhanov [2010a] for {0,∞}-valued languages containing all finite-valued unary cost functions (a.k.a. Min-Cost-Hom).We prove a Schaefer-like dichotomy theorem for conservative valued languages: if all cost functions in the language satisfy a certain condition (specified by a complementary combination ofSTP and MJN multimorphisms), then any instance can be solved in polynomial time (via a new algorithm developed in this article), otherwise the language is NP-hard. This is the first complete complexity classification ofgeneral-valued constraint languagesover non-Boolean domains. It is a common phenomenon that complexity classifications of problems over non-Boolean domains are significantly harder than the Boolean cases. The polynomial-time algorithm we present for the tractable cases is a generalization of the submodular minimization problem and a result of Cohen et al. [2008].Our results generalize previous results by Takhanov [2010a] and (a subset of results) by Cohen et al. [2006] and Deineko et al. [2008]. Moreover, our results do not rely on any computer-assisted search as in Deineko et al. [2008], and provide a powerful tool for proving hardness of finite-valued and general-valued languages.

Minimizing a sum of submodular functions

We consider the problem of minimizing a function represented as a sum of submodular terms. We assume each term allows an efficient computation of exchange capacities. This holds, for example, for terms depending on a small number of variables, or for certain cardinality-dependent terms.A naive application of submodular minimization algorithms would not exploit the existence of specialized exchange capacity subroutines for individual terms. To overcome this, we cast the problem as a submodular flow (SF) problem in an auxiliary graph in such a way that applying most existing SF algorithms would rely only on these subroutines.We then explore in more detail Iwata’s capacity scaling approach for submodular flows (Iwata 1997 [19]). In particular, we show how to improve its complexity in the case when the function contains cardinality-dependent terms.