Multilinear Extension Research Articles

Practical and Parallelizable Algorithms for Non-Monotone Submodular Maximization with Size Constraint

We present combinatorial and parallelizable algorithms for the maximization of a submodular function, not necessarily monotone, with respect to a size constraint. We improve the best approximation factor achieved by an algorithm that has optimal adaptivity and nearly optimal query complexity to 1/6 − ε, and even further to 0.193 − ε by increasing the adaptivity by a factor of O(log(k)). The conference version of this work mistakenly employed a subroutine that does not work for non-monotone, submodular functions. In this version, we propose a fixed and improved subroutine to add a set with high average marginal gain, ThreshSeq, which returns a solution in O(log(n)) adaptive rounds with high probability. Moreover, we provide two approximation algorithms. The first has approximation ratio 1/6 − ε, adaptivity O(log(n)), and query complexity O(n log(k)), while the second has approximation ratio 0.193 − ε, adaptivity O(log(n) log(k)), and query complexity O(n log(k)). Our algorithms are empirically validated to use a low number of adaptive rounds and total queries while obtaining solutions with high objective value in comparison with state-of-the-art approximation algorithms, including continuous algorithms that use the multilinear extension.

Large deviations for mean field model in Erdős–Rényi graph

In this paper, we study a particle systems (or interacting diffusions) on an Erdős–Rényi graph with the parameter pN∈(0,1] that behaves like a mean-field system up to large deviations. Our aim is to establish the large deviations for the empirical measure process of particle systems under the condition NpN4→∞ as N→∞, where N is the number of particles. The result is obtained by proving the exponential equivalence between our systems and general interacting systems without random graphs. The multilinear extensions of Grothendieck inequality play a crucial role in our proof.

Distributed strategy selection: A submodular set function maximization approach

Joint utility-maximization problems for multi-agent systems often should be addressed by distributed strategy-selection formulation. Constrained by discrete feasible strategy sets, these problems are broadly formulated as NP-hard combinatorial optimization problems. In many cases, these problems can be cast as constrained submodular set function maximization problems, which also belong to the NP-hard domain of problems. A prominent example is the problem of multi-agent mobile sensor dispatching over a discrete domain. This paper considers a class of submodular optimization problems that consist of maximization of a monotone and submodular set function subject to a partition matroid constraint over a group of networked agents that communicate over a connected undirected graph. We work with the value oracle model. Consequently, the only access of the agents to the utility function is through a black box that returns the utility function value given a specific strategy set. We propose a distributed suboptimal polynomial-time algorithm that enables each agent to obtain its respective strategy via local interactions with its neighboring agents. Our solution is a fully distributed gradient-based algorithm using the submodular set functions’ multilinear extension followed by a distributed stochastic Pipage rounding procedure. This algorithm results in a strategy set that when the team utility function is evaluated at the worst case, the utility function value is in 1c(1−e−c−O(1/T)) of the optimal solution with c being the curvature of the submodular function. An example demonstrates our results.

Properties and Algorithm of Lattice Pseudo-Submodular Functions

In this paper, we propose a concept of a lattice pseudo-submodular (LPS) function and consider maximizing a monotone continuous real LPS function [Formula: see text] under a convex polytope constraint. The concept of LPS function was proposed to describe the properties of some discrete functions or nonconvex continuous functions. It is a generalization of the lattice submodular function. For the real LPS maximization problem, we design the monotone Pseudo Frank-Wolfe (PFW) algorithm by taking advantage of the second derivative bound. The PFW algorithm iterates by constantly optimize linear gradient function [Formula: see text], and finally outputs the solution. We theoretically prove that PFK algorithm has an approximation ratio of [Formula: see text] (where [Formula: see text]), and it needs at least [Formula: see text] rounds (where [Formula: see text] is a parameter given in advance). The PFW algorithm is also useful for multilinear extension of discrete lattice pseudo-submodular maximization problems.

Jacobi-Style Iteration for Distributed Submodular Maximization

This article presents a novel Jacobi-style iteration algorithm for solving the problem of distributed submodular maximization, in which multiple agents determine their strategies from the private sets so that a global, nonseparable submodular objective function is jointly maximized. Building on the multilinear extension of the submodular function, we expect to achieve the solution from a probabilistic, rather than deterministic, perspective, and thus, transfer the considered problem from the discrete domain into the continuous domain. Since it is observed that an unbiased estimation of the gradient of multilinear extension function can be obtained by sampling the agents’ local strategies, a projected stochastic gradient algorithm is proposed to solve the problem. Our algorithm enables simultaneous updates among all individual agents and guarantees to converge asymptotically to a desirable equilibrium solution. Such an equilibrium solution is shown to be at least <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1/2$</tex-math></inline-formula> suboptimal, which is comparable to the state-of-art in the literature. The convergence rate, which is characterized by the running average of gradient mapping, is proved to be <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1/T)$</tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$T$</tex-math></inline-formula> is the number of iterations. Moreover, we further enhance the proposed algorithm by handling the scenario in which agents’ communication delays are present. The enhanced algorithm admits a more realistic distributed implementation of our approach. Finally, a movie recommendation task is conducted on a real-world movie rating dataset to validate the numerical performance of our algorithms.

Möbius product-based constructions of aggregation functions

Möbius transforms of capacities or games were considered as a tool for extensions of capacities to particular aggregation functions in several papers. This is, for example, the case of the Lovász extension coinciding with the Choquet integral, or the Owen extension that is also known as multilinear extension. In this paper, a much deeper study of the links between the Möbius transforms of real-valued functions defined on finite bounded posets and aggregation functions is performed. We introduce a Möbius product of any two real-valued functions defined on an arbitrary finite bounded poset, and then, fixing the poset (2N,⊆), N={1,…,n}, we propose and discuss a construction method for n-ary aggregation functions based on the Möbius product of any capacity on N and a real-valued function gx, x∈[0,1]n, defined on 2N and determined by some appropriate n-ary aggregation function. We provide some necessary and some sufficient conditions for the introduced construction to yield an aggregation function for any capacity. For the binary case, a complete characterization of conditions under which our approach results in an aggregation function for each capacity m is given.

Log concavity and concentration of Lipschitz functions on the Boolean hypercube

It is well-known that measures whose density is the form e−V where V is a uniformly convex potential on Rn attain strong concentration properties. In search of a notion of log-concavity on the discrete hypercube, we consider measures on {−1,1}n whose multi-linear extension f satisfies log⁡∇2f(x)⪯βIn, for β≥0, which we refer to as β-semi-log-concave. We prove that these measures satisfy a nontrivial concentration bound, namely, any Hamming Lipchitz test function φ satisfies Varν[φ]≤n2−Cβ for Cβ>0. As a corollary, we prove a concentration bound for measures which exhibit the so-called Rayleigh property. Namely, we show that for measures such that under any external field (or exponential tilt), the correlation between any two coordinates is non-positive, Hamming-Lipschitz functions admit nontrivial concentration.

Weak null, necessary defender and necessary detractor players: characterizations of the Banzhaf and the Shapley bisemivalues

We focus on bicooperative games, a variation of the classic cooperative games and, in particular, on the Banzhaf and the Shapley bisemivalues defined on these games. We consider three special classes of players: weak null, necessary defender and necessary detractor players. By introducing new properties related to this kind of players, we provide new axiomatic characterizations of the Banzhaf and the Shapley bisemivalues giving, in both cases, a set of independent properties that univocally determine them. We also provide a computational procedure to calculate the allocations given by the Shapley bisemivalue in terms of the generalized multilinear extension of the game.

A Monotonic Weighted Banzhaf Value for Voting Games

The aim of this paper is to extend the classical Banzhaf index of power to voting games in which players have weights representing different cooperation or bargaining abilities. The obtained value does not satisfy the classical total power property, which is justified by the imperfect cooperation. Nevertheless, it is monotonous in the weights. We also obtain three different characterizations of the value. Then we relate it to the Owen multilinear extension.

Nearly Linear-Time, Parallelizable Algorithms for Non-Monotone Submodular Maximization

We study combinatorial, parallelizable algorithms for maximization of a submodular function, not necessarily monotone, with respect to a cardinality constraint k. We improve the best approximation factor achieved by an algorithm that has optimal adaptivity and query complexity, up to logarithmic factors in the size of the ground set, from 0.039 to nearly 0.193. Heuristic versions of our algorithms are empirically validated to use a low number of adaptive rounds and total queries while obtaining solutions with high objective value in comparison with state-of-the-art approximation algorithms, including continuous algorithms that use the multilinear extension.

Nonadditive integration and some solutions of cooperative games

In the paper, we propose three schemes of nonadditive integration based on several extensions of nonadditive set function and integrand to the appropriate symmetric power of the original measurable space. A survey on the integral representation of some classic objects of the cooperative game theory, derived by nonadditive integration, is given. A universal approach for investigation of both finite and infinite games is developed. We pay a particular attention to the Shapley value, Owen multilinear extension, and support function of the core of a convex cooperative game.

Parallelized maximization of nonsubmodular function subject to a cardinality constraint

In the paper, we consider the problem of maximizing the multilinear extension of a nonsubmodular set function subject to a k-cardinality constraint with adaptive rounds of evaluation queries. We devise an algorithm which achieves a ratio of (1−e−γ2−ϵ) and requires O(log⁡n/ϵ2) adaptive rounds and O(nlog⁡n/ϵ2) queries, where γ is the continuous generic submodularity ratio that compares favorably in flexibility to the traditional submodularity ratio proposed by Das and Kempe. The key idea of our algorithm is originated from the parallel-greedy algorithm proposed by Chekuri et al., but incorporating with two major changes to retain the performance guarantee: First, identify all good coordinates with the continuous generic submodularity ratio and gradient values approximately as large as the best coordinate, and increase along all these coordinates uniformly; Second, increase x along these coordinates by a dynamical increment whose value depends on γ. The key difficulty of our algorithm is that when the function is nonsubmodular, the set of the best coordinate does not decrease during iterations; while provided submodularity, the decreasing can be ensured by the parallel-greedy algorithm. Our algorithms slightly compromise performance guarantee for the sake of extending to constrained nonsubmodular maximization with parallelism, provided that the state-of-art algorithm for the corresponding submodular version attains an approximation ratio of (1−1/e−ϵ) and requires O(log⁡n/ϵ2) adaptive rounds.

Budgeted Coupon Advertisement Problem: Algorithm and Robust Analysis

Coupon advertisement is everywhere in people's daily lives, and it is a common marketing strategy adopted by merchants. A problem, Budget Profit Maximization with Coupon Advertisement, is formulated in this article, which is a branch of classical profit maximization problem in social networks. Profit maximization has been researched intensively before, but its theoretical bound is not satisfactory usually because of NP-hardness, budget constraint and non-monotonicity. Learned from the lastest results, we proposed the B-Framework, which combines the ideas of Random Greedy and Continuous Double Greedy to obtain a more acceptable approximation ratio for this problem. For Continuous Double Greedy, it can be implemented by multilinear extension and discretized techniques. In addition, most of existing researches consider only maximizing total profit, however, in real scenarios, the diffusion probability is hard to determine due to the uncertainty. Then, we study the robustness for budgeted profit maximization, which can be used as a general strategy to analyze the robustness of non-monotone submodular function. It aims to obtain the best solution maximizing the ratio between the profit of any feasible seed set and the optimal seed set. We design LU-B-Framework first, and then we apply the method of uniform sampling to improve the robustness by reducing the uncertainty. The effectiveness and correctness of our algorithms are evaluated by heavy simulation on real-world social networks eventually.

Some Properties for Bisemivalues on Bicooperative Games

In this work, we focus on bicooperative games, a variation of the classic cooperative games, and investigate the conditions for the coefficients of the bisemivalues—a generalization of semivalues for cooperative games—necessary and / or sufficient in order to satisfy some properties, including among others, desirability relation, balanced contributions, null player exclusion property and block property. Moreover, a computational procedure to calculate bisemivalues in terms of the multilinear extension of the game is given.

Constrained Submodular Maximization via a Nonsymmetric Technique

The study of combinatorial optimization problems with submodular objectives has attracted much attention in recent years. Such problems are important in both theory and practice because their objective functions are very general. Obtaining further improvements for many submodular maximization problems boils down to finding better algorithms for optimizing a relaxation of them known as the multilinear extension. In this work, we present an algorithm for optimizing the multilinear relaxation whose guarantee improves over the guarantee of the best previous algorithm (by Ene and Nguyen). Moreover, our algorithm is based on a new technique that is, arguably, simpler and more natural for the problem at hand. In a nutshell, previous algorithms for this problem rely on symmetry properties that are natural only in the absence of a constraint. Our technique avoids the need to resort to such properties, and thus seems to be a better fit for constrained problems.

A modified position value for communication situations and its fuzzification

Myerson treated various situations of cooperation in the field of cooperative games and proposed the communication structure. In this paper, we define and characterize an allocation rule in terms of the position value, called an average spanning tree solution, for communication situations by introducing a concept of cooperation relationship which says that two players are deemed to possess this relationship if there is a communication path through them. Considering the fact that the extent to what players participate in a coalition may be partially restricted for uncertain possibilities, we construct a graph game in multilinear extension form and continue to explore the fuzzy average spanning tree solution on a framework of communication situation with fuzzy coalition.

Probabilistic Rank-One Tensor Analysis With Concurrent Regularizations.

Subspace learning for tensors attracts increasing interest in recent years, leading to the development of multilinear extensions of principal component analysis (PCA) and probabilistic PCA (PPCA). Existing multilinear PPCAs are based on the Tucker or CANDECOMP/PARAFAC (CP) models. Although both kinds of multilinear PPCAs have shown their effectiveness in dealing with tensors, they also have their own limitations. Tucker-based multilinear PPCAs have a restrictive subspace representation and suffer from rotational ambiguity, while CP-based ones are more prone to overfitting. To address these problems, we propose probabilistic rank-one tensor analysis (PROTA), a CP-based multilinear PPCA. PROTA has a more flexible subspace representation than Tucker-based PPCAs, and avoids rotational ambiguity. To alleviate overfitting for CP-based PPCAs, we propose two simple and effective regularization strategies, named as concurrent regularizations (CRs). By adjusting the noise variance or the moments of latent features, our strategies concurrently and coherently penalize the entire subspace. This relaxes unnecessary scale restrictions and gains more flexibility in regularizing CP-based PPCAs. To take full advantage of the probabilistic framework, we further propose a Bayesian treatment of PROTA, which achieves both automatic feature determination and robustness against overfitting. Experiments on synthetic and real-world datasets demonstrate the superiority of PROTA in subspace estimation and classification, as well as the effectiveness of CRs in alleviating overfitting.

Tensor-based sparse representations of multi-phase medical images for classification of focal liver lesions

Medical images play an important role in clinics. In most clinic sites, the diagnosis of diseases and the comprehending of disease progression need firstly accurate interpretation of the available medical images, which would be time-consuming in manual interpretation of the accumulated large amount of medical images. Thus automatical analysis and understanding of the available medical images become an active research topic, and therein, feature extraction of medical images plays an important role for achieving diagnosis performance. Natural images hold two-dimensional structures and linear statistical methods, such as k-means, GMM, and sparse coding, are widely applied for extracting compact and inherent representations. In contrast, medical images themselves have three-dimensional structures and possibly consist of multi-phase extension. Directly applying linear methods on the available medical data would lead to high dimensional vectors by reshaping the multi-dimensional data and would destroy the correlated relation among different dimensions of the raw medical data domain. Therefore, this study proposes a multilinear extension of the linear sparse coding for extracting compact and effective intermediate representations of the multi-dimensional local structures in multi-phase CT images, and aggregating the intermediate representations in the Bag-of-Visual-Words (BoVW) manner for classification of focal liver lesions (FLLs). In the proposed approach, three-layer volumes from the corresponding slices of multi-phase CT images are formed and spatiotemporal local structures from the volumes are extracted as 3rd-order tensors. Regarding the high dimensional local structures as tensors, we propose a K-CP (CANDECOMP/PARAFAC) algorithm to learn a tensor dictionary in an iterative way and extract the sparse representation with a multilinear OMP method. The aggregation of the sparse representation is implemented in the BoVW manner, which has been proved to be an effective method for extracting features from natural images and medical images. The proposed strategy is evaluated in classification of focal liver lesions and achieved better results than conventional BoVW with linear statistical methods.

Unsupervised classification of multichannel profile data using PCA: An application to an emission control system

Abstract Modern sensing technologies have facilitated real-time data collection for process monitoring and fault diagnosis in several research fields of industrial engineering. The challenges associated with diagnosis of multichannel (multiple) profiles are yet to be addressed in the literature. Motivated by an application of fault diagnosis of an emission control system, this paper proposes an approach for efficient and interpretable modeling of multichannel profile data in high-dimensional spaces. The method is based on unsupervised classification of multichannel profile data provided by several sensors related to a fault event. The final goal is to isolate fault events in a restricted number of clusters (scenarios), each one described by a reference pattern. This can provide practitioners with useful information to support the diagnosis and to find root cause. Two multilinear extensions of principal component analysis (PCA), which can analyze the multichannel profiles without unfolding the original data set, are investigated and compared to regular PCA applied to vectors generated by unfolding the original data set. The effectiveness of multilinear extensions of PCA is demonstrated using an experimental campaign carried out on an emission control system. Results of unsupervised classification show that the multilinear extension of PCA may lead to a classification with better compactness and separation of clusters.

Distributed geometric nonnegative matrix factorization and hierarchical alternating least squares–based nonnegative tensor factorization with the MapReduce paradigm

SummaryNonnegative matrix factorization and its multilinear extension known as nonnegative tensor factorization are commonly used methods in machine learning and data analysis for feature extraction and dimensionality reduction for nonnegative high‐dimensional data. Dimensionality reduction for massive amounts of data usually involves distributed computation across multi‐node computer architectures. In this study, we propose various computational strategies for parallel and distributed computation of the latent factors in both factorization models, all of which are based on partitioning the computational tasks according to the MapReduce paradigm. We extend the previously reported distributed hierarchical alternating least squares algorithm to the multi‐way array factorization model, where we assume that the observed multi‐way data can be partitioned into chunks along one mode. Moreover, we propose a new geometry‐based distributed computational strategy for solving nonnegative matrix factorization problems. Numerical experiments performed using various large‐scale data sets demonstrated that these algorithms are efficient and robust to noisy data.