Multiplicative Weights Update Method Research Articles

Practically motivated adaptive fusion method with tie analysis for multilabel dispersed data

The goal of this paper is to propose an adaptive fusion method that aggregates the results of predictions obtained for a set of classifiers. The problem of selecting the best fusion method for a given data set is an important but difficult task. For different data sets, different fusion methods work best, however, it is not possible to predict in advance which method will be appropriate for a given data set. In the proposed adaptive method, the weights assigned to the fusion methods are adjusted iteratively. The proposed method is based on empirically developed formulas. The results obtained using this method were compared with the results obtained from the online learning method — the Multiplicative Weights approach, which is asymptotically optimal (Arora, S., Hazan, E., & Kale, S., 2012). The multiplicative weights update method: a meta-algorithm and applications. Theory Comput., 8, 121–164). However, the results obtained with the use of the proposed method turns out to be better when applied to a finite set of test objects. In addition, a comparative analysis is provided when fusion methods are used simultaneously. Also, the results obtained when different fusion methods are used individually were compared. It was shown that the results obtained by the adaptive fusion method are much better.

Randomized Strategies for Robust Combinatorial Optimization

In this paper, we study the following robust optimization problem. Given an independence system and candidate objective functions, we choose an independent set, and then an adversary chooses one objective function, knowing our choice. The goal is to find a randomized strategy (i.e., a probability distribution over the independent sets) that maximizes the expected objective value in the worst case. This problem is fundamental in wide areas such as artificial intelligence, machine learning, game theory and optimization. To solve the problem, we propose two types of schemes for designing approximation algorithms. One scheme is for the case when objective functions are linear. It first finds an approximately optimal aggregated strategy and then retrieves a desired solution with little loss of the objective value. The approximation ratio depends on a relaxation of an independence system polytope. As applications, we provide approximation algorithms for a knapsack constraint or a matroid intersection by developing appropriate relaxations and retrievals. The other scheme is based on the multiplicative weights update (MWU) method. The direct application of the MWU method does not yield a strict multiplicative approximation algorithm but yield one with an additional additive error term. A key technique to overcome the issue is to introduce a new concept called (η,γ)-reductions for objective functions with parameters η and γ. We show that our scheme outputs a nearly α-approximate solution if there exists an α-approximation algorithm for a subproblem defined by (η,γ)-reductions. This improves approximation ratios in previous results. Using our result, we provide approximation algorithms when the objective functions are submodular or correspond to the cardinality robustness for the knapsack problem.

A Multiplicative Weight Updates Algorithm for Packing and Covering Semi-infinite Linear Programs

We consider the following semi-infinite linear programming problems: $$\max $$ (resp., $$\min $$ ) $$c^Tx$$ s.t. $$y^TA_ix+(d^i)^Tx \le b_i$$ (resp., $$y^TA_ix+(d^i)^Tx \ge b_i)$$ , for all $$y \in {{\mathcal {Y}}}_i$$ , for $$i=1,\ldots ,N$$ , where $${{\mathcal {Y}}}_i\subseteq {\mathbb {R}}^m_+$$ are given compact convex sets and $$A_i\in {\mathbb {R}}^{m_i\times n}_+$$ , $$b=(b_1,\ldots b_N)\in {\mathbb {R}}_+^N$$ , $$d^i\in {\mathbb {R}}_+^n$$ , and $$c\in {\mathbb {R}}_+^n$$ are given non-negative matrices and vectors. This general framework is useful in modeling many interesting problems. For example, it can be used to represent a sub-class of robust optimization in which the coefficients of the constraints are drawn from convex uncertainty sets $${{\mathcal {Y}}}_i$$ , and the goal is to optimize the objective function for the worst case choice in each $${{\mathcal {Y}}}_i$$ . When the uncertainty sets $${{\mathcal {Y}}}_i$$ are ellipsoids, we obtain a sub-class of second-order cone programming. We show how to extend the multiplicative weights update method to derive approximation schemes for the above packing and covering problems. When the sets $${{\mathcal {Y}}}_i$$ are simple, such as ellipsoids or boxes, this yields substantial improvements in running time over general convex programming solvers. We also consider the mixed packing/covering problem, in which both packing and covering constraints are given, and the objective is to find an approximately feasible solution.

Towards More Practical Linear Programming-based Techniques for Algorithmic Mechanism Design

R. Lavi and C. Swamy (FOCS 2005, J. ACM 58(6), 25, 2011) introduced a general method for obtaining truthful-in-expectation mechanisms from linear programming based approximation algorithms. Due to the use of the Ellipsoid method, a direct implementation of the method is unlikely to be efficient in practice. We propose to use the much simpler and usually faster multiplicative weights update method instead. The simplification comes at the cost of slightly weaker approximation and truthfulness guarantees.

Fast approximation for computing the fractional arboricity and extraction of communities of a graph

In this paper, we develop some algorithmic aspects of the fractional arboricity of a graph in order to study some new approaches for graph clustering. For a given undirected graph G(V,E) with m edges and n vertices, the fractional arboricity γ(G) measures the maximum edge density of the subgraphs of G(V,E). It is the fractional covering number of the corresponding graphic matroid. The fractional arboricity, for applications in networks reliability or the approximation of the chromatic number of the graphs, has been studied for many years. There are some algorithms in polynomial time to compute the fractional arboricity and its integer part. But, for large graphs such as the graph of the Web or graphs of social networks, the exact algorithms are not fast enough for practical use. That is why we describe a new FPTAS to compute an ε-approximation of the fractional arboricity (with ε>0 as small as desired). Our algorithm uses the principle of the multiplicative weights update method, needs a memory of size O(m) and has a complexity of O(mlog2(m)log(mn)/ε2). We also give a 2-approximation of γ(G) with computation time O(m), which is a quick preprocessing for the main algorithm. Finally, we present a fast algorithm to extract a subgraph which achieves the value of the approximation of the fractional arboricity.

Parallel Approximation of Min-Max Problems

This paper presents an efficient parallel approximation scheme for a new class of min-max problems. The algorithm is derived from the matrix multiplicative weights update method and can be used to find near-optimal strategies for competitive two-party classical or quantum interactions in which a referee exchanges any number of messages with one party followed by any number of additional messages with the other. It considerably extends the class of interactions which admit parallel solutions, demonstrating for the first time the existence of a parallel algorithm for an interaction in which one party reacts adaptively to the other. As a consequence, we prove that several competing-provers complexity classes collapse to PSPACE such as QRG(2), SQG and two new classes called DIP and DQIP. A special case of our result is a parallel approximation scheme for a specific class of semidefinite programs whose feasible region consists of lists of semidefinite matrices that satisfy a transcript-like consistency condition. Applied to this special case, our algorithm yields a direct polynomial-space simulation of multi-message quantum interactive proofs resulting in a first-principles proof of QIP=PSPACE.

The Multiplicative Weights Update Method: A Meta-Algorithm and Applications

Algorithms in varied fields use the idea of maintaining a distribution over a certain set and use the multiplicative update rule to iteratively change these weights. Their analyses are usually very similar and rely on an exponential potential function. In this survey we present a simple meta-algorithm that unifies many of these disparate algorithms and derives them as simple instantiations of the meta-algorithm. We feel that since this meta-algorithm and its analysis are so simple, and its applications so broad, it should be a standard part of algorithms courses, like “divide and conquer.”

QIP = PSPACE

This work considers the quantum interactive proof system model of computation, which is the (classical) interactive proof system model’s natural quantum computational analogue. An exact characterization of the expressive power of quantum interactive proof systems is obtained: the collection of computational problems having quantum interactive proof systems consists precisely of those problems solvable by deterministic Turing machines that use at most a polynomial amount of space (or, more succinctly, QIP = PSPACE). This characterization is proved through the use of a parallelized form of the matrix multiplicative weights update method, applied to a class of semidefinite programs that captures the computational power of quantum interactive proof systems. One striking implication of this characterization is that quantum computing provides no increase in computational power whatsoever over classical computing in the context of interactive proof systems, for it is well known that the collection of computational problems having classical interactive proof systems coincides with those problems solvable by polynomial-space computations.