Submodular Research Articles

Structured Robust Submodular Maximization: Offline and Online Algorithms

Constrained submodular function maximization has been used in subset selection problems such as selection of most informative sensor locations. Although these models have been quite popular, the solutions obtained via this approach are unstable to perturbations in data defining the submodular functions. Robust submodular maximization has been proposed as a richer model that aims to overcome this discrepancy as well as increase the modeling scope of submodular optimization. In this work, we consider robust submodular maximization with structured combinatorial constraints and give efficient algorithms with provable guarantees. Our approach is applicable to constraints defined by single or multiple matroids and knapsack as well as distributionally robust criteria. We consider both the offline setting where the data defining the problem are known in advance and the online setting where the input data are revealed over time. For the offline setting, we give a general (nearly) optimal bicriteria approximation algorithm that relies on new extensions of classical algorithms for submodular maximization. For the online version of the problem, we give an algorithm that returns a bicriteria solution with sublinear regret.Summary of Contribution: Constrained submodular maximization is one of the core areas in combinatorial optimization with a wide variety of applications in operations research and computer science. Over the last decades, both communities have been interested on the design and analysis of new algorithms with provable guarantees. Sensor location, influence maximization and data summarization are some of the applications of submodular optimization that lie at the intersection of the aforementioned communities. Particularly, our work focuses on optimizing several submodular functions simultaneously. We provide new insights and algorithms to the offline and online variants of the problem which significantly expand the related literature. At the same time, we provide a computational study that supports our theoretical results.

Approximation Algorithms for Submodular Data Summarization with a Knapsack Constraint

Data summarization, i.e., selecting representative subsets of manageable size out of massive data, is often modeled as a submodular optimization problem. Although there exist extensive algorithms for submodular optimization, many of them incur large computational overheads and hence are not suitable for mining big data. In this work, we consider the fundamental problem of (non-monotone) submodular function maximization with a knapsack constraint, and propose simple yet effective and efficient algorithms for it. Specifically, we propose a deterministic algorithm with approximation ratio 6 and a randomized algorithm with approximation ratio 4, and show that both of them can be accelerated to achieve nearly linear running time at the cost of weakening the approximation ratio by an additive factor of ε. We then consider a more restrictive setting without full access to the whole dataset, and propose streaming algorithms with approximation ratios of 8+ε and 6+ε that make one pass and two passes over the data stream, respectively. As a by-product, we also propose a two-pass streaming algorithm with an approximation ratio of 2+ε when the considered submodular function is monotone. To the best of our knowledge, our algorithms achieve the best performance bounds compared to the state-of-the-art approximation algorithms with efficient implementation for the same problem. Finally, we evaluate our algorithms in two concrete submodular data summarization applications for revenue maximization in social networks and image summarization, and the empirical results show that our algorithms outperform the existing ones in terms of both effectiveness and efficiency.

Revisiting Modified Greedy Algorithm for Monotone Submodular Maximization with a Knapsack Constraint

Monotone submodular maximization with a knapsack constraint is NP-hard. Various approximation algorithms have been devised to address this optimization problem. In this paper, we revisit the widely known modified greedy algorithm. First, we show that this algorithm can achieve an approximation factor of 0.405, which significantly improves the known factors of 0.357 given by Wolsey and (1-1/e)/2\approx 0.316 given by Khuller et al. More importantly, our analysis closes a gap in Khuller et al.'s proof for the extensively mentioned approximation factor of (1-1/\sqrte )\approx 0.393 in the literature to clarify a long-standing misconception on this issue. Second, we enhance the modified greedy algorithm to derive a data-dependent upper bound on the optimum. We empirically demonstrate the tightness of our upper bound with a real-world application. The bound enables us to obtain a data-dependent ratio typically much higher than 0.405 between the solution value of the modified greedy algorithm and the optimum. It can also be used to significantly improve the efficiency of algorithms such as branch and bound.

Evolutionary algorithms and submodular functions: benefits of heavy-tailed mutations

A core operator of evolutionary algorithms (EAs) is the mutation. Recently, much attention has been devoted to the study of mutation operators with dynamic and non-uniform mutation rates. Following up on this area of work, we propose a new mutation operator and analyze its performance on the \((1+1)\) Evolutionary Algorithm (EA). Our analyses show that this mutation operator competes with pre-existing ones, when used by the \((1+1)\) EA on classes of problems for which results on the other mutation operators are available. We show that the \((1+1)\) EA using our mutation operator finds a (1/3)-approximation ratio on any non-negative submodular function in polynomial time. We also consider the problem of maximizing a symmetric submodular function under a single matroid constraint and show that the \((1+1)\) EA using our operator finds a (1/3)-approximation within polynomial time. This performance matches that of combinatorial local search algorithms specifically designed to solve these problems and outperforms them with constant probability. Finally, we evaluate the performance of the \((1+1)\) EA using our operator experimentally by considering two applications: (a) the maximum directed cut problem on real-world graphs of different origins, with up to 6.6 million vertices and 56 million edges and (b) the symmetric mutual information problem using a four month period air pollution data set. In comparison with uniform mutation and a recently proposed dynamic scheme, our operator comes out on top on these instances.

Sensor Selection for Dynamics-Driven User-Interface Design

We present a method for dynamics-driven, user-interface design for a human–automation system via sensor selection. We define the user interface to be the output of a multiple-input–multiple-output (MIMO) linear time-invariant (LTI) system and formulate the design problem as one of selecting an output matrix from a given set of candidate output matrices. Necessary conditions for situation awareness are captured as additional constraints on the selection of the output matrix. These constraints depend on the level of trust the human has in the automation. We show that the resulting user-interface design problem is a combinatorial, set-cardinality minimization problem with set function constraints. We propose tractable algorithms to compute optimal or suboptimal solutions with suboptimality bounds. Our approaches exploit monotonicity and submodularity present in the design problem and rely on constraint programming and submodular maximization. We apply this method to the IEEE 118-bus, to construct correct-by-design interfaces under various operating scenarios.

Opinion subset selection via submodular maximization

Current research on subset selection for opinion analysis assumes that their methods can retrieve the opinions expressed in documents from general text features. However, such relaxed conditions can hardly maintain the performance of the analysis in opinion mining, especially when given strict limitations on the subset size. In this paper, we propose a framework for opinion subset selection. This framework can select a small set of instances from original data to convey a subjective representation for opinion classification and regression. Compared with our framework, the conventional submodular based subset selection approach cannot capture the fine-grained opinion features expressed in the corpus. Specifically, we propose a monotone non-decreasing score function and a framework based on topic modeling and submodular maximization for filtering irrelevant information and selecting the subsets. Our work further introduces an opinion-sensitive algorithm for optimizing the proposed function for opinion subset construction. We perform extensive experiments and comparative analysis of different subset selection methods in this work. The experimental result shows that the proposed opinion subset selection framework can compress the original text training set and preserve the test set’s classification and regression metric performance at the same time.

A sample decreasing threshold greedy-based algorithm for big data summarisation

As the scale of datasets used for big data applications expands rapidly, there have been increased efforts to develop faster algorithms. This paper addresses big data summarisation problems using the submodular maximisation approach and proposes an efficient algorithm for maximising general non-negative submodular objective functions subject to k-extendible system constraints. Leveraging a random sampling process and a decreasing threshold strategy, this work proposes an algorithm, named Sample Decreasing Threshold Greedy (SDTG). The proposed algorithm obtains an expected approximation guarantee of frac{1}{1+k}-epsilon for maximising monotone submodular functions and of frac{k}{(1+k)^2}-epsilon in non-monotone cases with expected computational complexity of Oleft(frac{n}{(1+k)epsilon }ln frac{r}{epsilon }right). Here, r is the largest size of feasible solutions, and epsilon in left(0, frac{1}{1+k}right) is an adjustable designing parameter for the trade-off between the approximation ratio and the computational complexity. The performance of the proposed algorithm is validated and compared with that of benchmark algorithms through experiments with a movie recommendation system based on a real database.

Trees of tangles in abstract separation systems

We prove canonical and non-canonical tree-of-tangles theorems for abstract separation systems that are merely structurally submodular. Our results imply all known tree-of-tangles theorems for graphs, matroids and abstract separation systems with submodular order functions, with greatly simplified and shortened proofs.

Shifting Opinions in a Social Network Through Leader Selection

We study the French-DeGroot opinion dynamics in a social network with two polarizing parties. We consider a network in which the leaders of one party are given, and we pose the problem of selecting the leader set of the opposing party so as to shift the average opinion to a desired value. When each party has only one leader, we express the average opinion in terms of the transition matrix and the stationary distribution of random walks in the network. The analysis shows balance of influence between the two leader nodes. We show that the problem of selecting at most $k$ absolute leaders to shift the average opinion is $\mathbf{NP}$-hard. Then, we reduce the problem to a problem of submodular maximization with a submodular knapsack constraint and an additional cardinality constraint and propose a greedy algorithm with upper bound search to approximate the optimum solution. We also conduct experiments in random networks and real-world networks to show the effectiveness of the algorithm.

Learning adaptive criteria weights for active semi-supervised learning

Batch mode active learning (BMAL) is devoted to training trustful learning models with scarce labeled samples by efficiently asking the ground truth annotations of the most beneficial unlabeled points for supervision with the feedback of an expert. Particularly, BMAL algorithms always sample points based on the decent-designed criteria, such as (un)certainty and representativeness, etc. However, present BMAL approaches consistently are afflicted with one limitation: They simply integrate the sampling criteria with fixed weights to select instances for supervised training, which may yield suboptimal batch acquisition since the criteria values of the plentiful candidate unlabeled samples would fluctuate after retraining the classifier with the newly augmented training set. Instead, the weights of sampling criteria should be allocated appropriately. To overcome this problem, this work proposes a novel Adaptive Criteria Weights batch selection algorithm, abbreviated ACW, which dynamically adjusts the importance of (un)certainty and representativeness to choose critical instances for semi-supervised learning. A submodular function is employed to recognize a diverse mini-batch from the selected batch of samples. We apply our proposed ACW batch sampling algorithm to two types of essential semi-supervised tasks, i.e., semi-supervised classification and semi-supervised clustering. To the best of our knowledge, this work is the first devoted attempt to explore adaptive mechanism of criteria weights in the context of active learning. The superiority and effectiveness of ACW against the present state-of-the-art BMAL approaches have also been demonstrated by the encouraging experimental results.

Simple and Efficient Budget Feasible Mechanisms for Monotone Submodular Valuations

We study the problem of a budget limited buyer who wants to buy a set of items, each from a different seller, to maximize her value. The budget feasible mechanism design problem requires the design a mechanism that incentivizes the sellers to truthfully report their cost and maximizes the buyer’s value while guaranteeing that the total payment does not exceed her budget. Such budget feasible mechanisms can model a buyer in a crowdsourcing market interested in recruiting a set of workers (sellers) to accomplish a task for her. This budget feasible mechanism design problem was introduced by Singer in 2010. We consider the general case where the buyer’s valuation is a monotone submodular function. There are a number of truthful mechanisms known for this problem. We offer two general frameworks for simple mechanisms, and by combining these frameworks, we significantly improve on the best known results, while also simplifying the analysis. For example, we improve the approximation guarantee for the general monotone submodular case from 7.91 to 5 and for the case of large markets (where each individual item has negligible value) from 3 to 2.58. More generally, given an r approximation algorithm for the optimization problem (ignoring incentives), our mechanism is a r + 1 approximation mechanism for large markets, an improvement from 2 r 2 . We also provide a mechanism without the large market assumption, where we achieve a 4 r + 1 approximation guarantee. We also show how our results can be used for the problem of a principal hiring in a Crowdsourcing Market to select a set of tasks subject to a total budget.

Submodular function minimization and polarity

Using polarity, we give an outer polyhedral approximation for the epigraph of set functions. For a submodular function, we prove that the corresponding polar relaxation is exact; hence, it is equivalent to the Lovász extension. The polar approach provides an alternative proof for the convex hull description of the epigraph of a submodular function. Computational experiments show that the inequalities from outer approximations can be effective as cutting planes for solving submodular as well as non-submodular set function minimization problems.

Improving in-memory file system reading performance by fine-grained user-space cache mechanisms

Nowadays, as the memory capacity of servers become larger and larger, distributed in-memory file systems, which enable applications to interact with data at fast speed, have been widely used. However, the existing distributed in-memory file systems still face the problem of low data access performance in small data reading, which seriously reduce their usefulness in many important big data scenarios. In this paper, we analyze the factors that affect the performance of reading in-memory files and propose a two-layer user space cache management mechanism: in the first layer, we cache data packet references to reduce frequent page fault interruptions (packet-level cache); in the second layer, we cache and manage small file data units to avoid redundant inter-process communications (object-level cache). We further design a fine-grained caching model based on the submodular function optimization theory, for efficiently managing the variable-length cache units with partially overlapping fragments on the client side. Experimental results on synthetic and real-world workloads show that compared with the existing cutting-edge systems, the first level cache can double the reading performance on average, and the second level cache can improve random reading performance by more than 4 times. Our caching strategies also outperform the cutting-edge cache algorithms over 20% on hit ratio. Furthermore, the proposed client-side caching framework idea has been adopted by the Alluxio open source community, which shows the practical benefits of this work. • A packet-level cache policy for reducing page faults during sequential data reading. • An in-memory cache layer to cache the variable-length hot file fragments. • Proved that the variable-length cache problem is a submodular optimization problem theoretically. • Comprehensive performance evaluation of the client-side cache strategies.

Online algorithms for BP functions maximization

BP maximization problem has many applications in machine learning and data science. It can be described as maximizing the sum of a suBmodular function and a suPermodular function (BP) under some constraints, where both functions are nonnegative and monotonic. In this paper, we consider two online cases. The first is a BP maximization problem subject to a uniform matroid constraint when the items arrive one-by-one, for which we offer an online algorithm with constant competitive ratio. The second is a BP maximization problem subject to a partition matroid constraint where each part of the partition arrives in a random order, for which we present two approximation algorithms of both constant competitive ratios, where one is randomized and the other is deterministic.

Non-submodular model for group profit maximization problem in social networks

In social networks, there exist many kinds of groups in which people may have the same interests, hobbies, or political orientation. Sometimes, group decisions are made by simply majority, which means that most of the users in this group reach an agreement, such as US Presidential Elections. A group is called activated if beta percent of users are influenced in the group. Enterprise will gain income from all influenced groups. Simultaneously, to propagate influence, enterprise needs pay advertisement diffusion cost. Group profit maximization (GPM) problem aims to pick k seeds to maximize the expected profit that considers the benefit of influenced groups with the diffusion cost. GPM is proved to be NP-hard and the objective function is proved to be neither submodular nor supermodular. An upper bound and a lower bound which are difference of two submodular functions are designed. We propose a submodular–modular algorithm (SMA) to solve the difference of two submodular functions and SMA is shown to converge to a local optimal. We present an randomized algorithm based on weighted group coverage maximization for GPM and apply sandwich framework to get theoretical results. Our experiments verify the efficiency of our methods.

Improved Streaming Algorithms for Maximizing Monotone Submodular Functions under a Knapsack Constraint

In this paper, we consider the problem of maximizing a monotone submodular function subject to a knapsack constraint in a streaming setting. In such a setting, elements arrive sequentially and at any point in time, and the algorithm can store only a small fraction of the elements that have arrived so far. For the special case that all elements have unit sizes (i.e., the cardinality-constraint case), one can find a \((0.5-\varepsilon )\)-approximate solution in \(O(K\varepsilon ^{-1})\) space, where K is the knapsack capacity (Badanidiyuru et al. KDD 2014). The approximation ratio is recently shown to be optimal (Feldman et al. STOC 2020). In this work, we propose a \((0.4-\varepsilon )\)-approximation algorithm for the knapsack-constrained problem, using space that is a polynomial of K and \(\varepsilon \). This improves on the previous best ratio of \(0.363-\varepsilon \) with space of the same order. Our algorithm is based on a careful combination of various ideas to transform multiple-pass streaming algorithms into a single-pass one.

Improved Randomized Algorithm for k-Submodular Function Maximization

Submodularity is one of the most important properties in combinatorial optimization, and $k$-submodularity is a generalization of submodularity. Maximization of a $k$-submodular function requires an exponential number of value oracle queries, and approximation algorithms have been studied. For unconstrained $k$-submodular maximization, Iwata, Tanigawa, and Yoshida, [Improved approximation algorithms for $k$-submodular function maximization, in Proceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, Philadelphia, 2016, pp. 404--413] gave a randomized $k/(2k-1)$-approximation algorithm for monotone functions and a randomized 1/2-approximation algorithm for nonmonotone functions. In this paper, we present improved randomized algorithms for nonmonotone functions. Our algorithm gives a $\frac{k^2+1}{2k^2+1}$-approximation for $k\geq 3$. We also give a randomized $\frac{\sqrt{17}-3}{2}$-approximation algorithm for $k=3$. We use the same framework used in Iwata, Tanigawa, and Yoshida, [Improved approximation algorithms for $k$-submodular function maximization, in Proceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, Philadelphia, 2016, pp. 404--413] and Ward and Živný [ACM Trans. Algorithms, 12 (2016), pp. 46:1--47:26] with different probabilities.

Online Contention Resolution Schemes with Applications to Bayesian Selection Problems

We introduce a new rounding technique designed for online optimization problems, which is related to contention resolution schemes, a technique initially introduced in the context of submodular function maximization. Our rounding technique, which we call online contention resolution schemes (OCRSs), is applicable to many online selection problems, including Bayesian online selection, oblivious posted pricing mechanisms, and stochastic probing models. It allows for handling a wide set of constraints and shares many strong properties of offline contention resolution schemes. In particular, OCRSs for different constraint families can be combined to obtain an OCRS for their intersection. Moreover, we can approximately maximize submodular functions in the online settings we consider. We thus get a broadly applicable framework for several online selection problems, which improves on previous approaches in terms of the types of constraints that can be handled, the objective functions that can be dealt with, and the assumptions on the strength of the adversary. Furthermore, we resolve two open problems from the literature; namely, we present the first constant-factor constrained oblivious posted price mechanism for matroid constraints and the first constant-factor algorithm for weighted stochastic probing with deadlines.

Discrete Signal Processing with Set Functions

Set functions are functions (or signals) indexed by the powerset (set of all subsets) of a finite set N. They are fundamental and ubiquitous in many application domains and have been used, for example, to formally describe or quantify loss functions for semantic image segmentation, the informativeness of sensors in sensor networks the utility of sets of items in recommender systems, cooperative games in game theory, or bidders in combinatorial auctions. In particular, the subclass of submodular functions occurs in many optimization and machine learning problems. In this paper, we derive discrete-set signal processing (SP), a novel shift-invariant linear signal processing framework for set functions. Discrete-set SP considers different notions of shift obtained from set union and difference operations. For each shift it provides associated notions of shift-invariant filters, convolution, Fourier transform, and frequency response. We provide intuition for our framework using the concept of generalized coverage function that we define, identify multivariate mutual information as a special case of a discrete-set spectrum, and motivate frequency ordering. Our work brings a new set of tools for analyzing and processing set functions, and, in particular, for dealing with their exponential nature. We show two prototypical applications and experiments: compression in submodular function optimization and sampling for preference elicitation in combinatorial auctions.

Robust Mixed 0-1 Programming and Submodularity

The paper studies a robust mixed integer program with a single unrestricted continuous variable. The purpose of the paper is the polyhedral study of the robust solution set using submodularity. A submodular function is a set function with a diminishing returns property, and little work has been studied on the utilization of submodularity in the study of optimization problems considering data uncertainty. In this paper, we propose valid inequalities using submodularity. Valid inequalities for the robust mixed integer program are defined. A polynomial separation algorithm is proposed, and we show that the convex hull of the problem can be completely described using the proposed inequalities. In computational tests, we showed the proposed cuts are effective when they are applied to general robust discrete optimization problems with one or multiple constraints.