Depth-3 Arithmetic Circuits Research Articles

On the Power of Border of Depth-3 Arithmetic Circuits

We show that over the field of complex numbers, every homogeneous polynomial of degree d can be approximated (in the border complexity sense) by a depth-3 arithmetic circuit of top fan-in at most 2. This is quite surprising, since there exist homogeneous polynomials P on n variables of degree 2, such that any depth-3 arithmetic circuit computing P must have top fan-in at least Ω ( n ). As an application, we get a new tradeoff between the top fan-in and formal degree in an approximate analog of the celebrated depth reduction result of Gupta, Kamath, Kayal, and Saptharishi [7, 10]. Formally, we show that if a degree d homogeneous polynomial P can be computed by an arithmetic circuit of size s ≥ d , then for every t ≤ d , P is in the border of a depth-3 circuit of top fan-in s O( t ) and formal degree s O( d/t ) . To the best of our knowledge, the upper bound on the top fan-in in the original proof of Reference [7] is always at least s Ω (√d) , regardless of the formal degree.

Special Section on the Forty-Third Annual ACM Symposium on Theory of Computing (STOC 2011)

This section of SIAM Journal on Computing contains extended versions of selected papers from the 43rd ACM Symposium on Theory of Computing (STOC), held June 6--8, 2011, in San Jose, California, as part of the fifth Federated Computing Research Conference (FCRC). The STOC proceedings contained 84 papers, which were selected from 304 submissions by the program committee, consisting of Ittai Abraham, Alexandr Andoni, Avrim Blum, Allan Borodin, Kousha Etessami, Lisa Fleischer, Venkatesan Guruswami, David Kempe, Frederic Magniez, Dieter van Melkebeek, Daniele Micciancio, Moni Naor, Kobbi Nissim, Seth Pettie, Ronitt Rubinfeld, Amir Shpilka, Ravi Sundaram, Eva Tardos, Prasad Tetali, Salil Vadhan (chair), Kasturi Varadarajan, Nisheeth Vishnoi, John Watrous, and Ryan Williams. Five of the STOC papers appear in this special section, each one expanded and fully refereed according to the high standards of the journal. They cover a diverse collection of topics: In “Distributed Verification and Hardness of Distributed Approximation,” Das Sarma, Holzer, Kor, Korman, Nanongkai, Pandurangan, Peleg, and Wattenhofer prove strong lower bounds on the power of distributed networks to verify their own properties (such as connectivity) and solve optimization problems such as computing approximate shortest paths or approximate min-cuts. They establish new connections between distributed computation and two-party communication complexity. The paper “Pareto Optimal Solutions for Smoothed Analysts” by Moitra and O'Donnell considers the smoothed complexity of discrete multi-objective optimization problems with $d+1$ linear objectives and with a solution space consisting of binary $n$-vectors. The authors show that, in a suitable smoothed analysis framework for such problems, the expected number of Pareto optimal solutions is at most $n^{2d}$. This improves greatly, as a function of the dimension d, an earlier upper bound established by Roeglin and Teng, which had roughly the form $n^{d^d}$. The paper “Blackbox Identity Testing for Bounded Top-Fanin Depth-3 Circuits: The Field Doesn't Matter” by Saxena and Seshadhri provides the first deterministic polynomial-time identity test for depth-3 arithmetic circuits with bounded top-fanin that only needs blackbox access to the circuit. Their construction has the feature that it works for arbitrary fields. In their paper “An Optimal Lower Bound on the Communication Complexity of Gap-Hamming-Distance,” Chakrabarti and Regev prove a lower bound establishing that the randomized communication complexity of the gap-Hamming-distance problem is linear. In obtaining this result, they have resolved an important and well-studied communication complexity problem having a fundamental connection to the data stream model of computation. Svensson's paper “Santa Claus Schedules Jobs on Unrelated Machines” breaks the barrier of 2 for efficiently approximating the minimum makespan for scheduling jobs on unrelated machines in the setting where all machines on which a given job can run take the same amount of time for that job. We thank the authors, the referees, and the full program committee for all their work, which made this special section possible.

Black box polynomial identity testing of generalized depth-3 arithmetic circuits with bounded top fan-in

In this paper we consider the problem of determining whether an unknown arithmetic circuit, for which we have oracle access, computes the identically zero polynomial. This problem is known as the black-box polynomial identity testing (PIT) problem. Our focus is on polynomials that can be written in the form $f(\bar x) = \sum\nolimits_{i = 1}^k {h_i (\bar x) \cdot g_i (\bar x)} $, where each h i is a polynomial that depends on only ρ linear functions, and each g i is a product of linear functions (when h i = 1, for each i, then we get the class of depth-3 circuits with k multiplication gates, also known as ΣΠΣ(k) circuits, but the general case is much richer). When maxi (deg(h i · g i )) = d we say that f is computable by a ΣΠΣ(k; d;ρ) circuit. We obtain the following results. A deterministic black-box identity testing algorithm for ΣΠΣ(k; d;ρ) circuits that runs in quasi-polynomial time (for ρ=polylog(n+d)). In particular this gives the first black-box quasi-polynomial time PIT algorithm for depth-3 circuits with k multiplication gates. A deterministic black-box identity testing algorithm for read-k ΣΠΣ circuits (depth-3 circuits where each variable appears at most k times) that runs in time $n^{2^{O(k^2 )} } $. In particular this gives a polynomial time algorithm for k=O(1). Our results give the first sub-exponential black-box PIT algorithm for circuits of depth higher than 2. Another way of stating our results is in terms of test sets for the underlying circuit model. A test set is a set of points such that if two circuits get the same values on every point of the set then they compute the same polynomial. Thus, our first result gives an explicit test set, of quasi-polynomial size, for ΣΠΣ(k; d;ρ) circuits (when ρ=polylog(n+d)). Our second result gives an explicit polynomial size test set for read-k depth-3 circuits. The proof technique involves a construction of a family of affine subspaces that have a rank-preserving property that is inspired by the construction of linear seeded extractors for affine sources of Gabizon and Raz [9], and a generalization of a theorem of [8] regarding the structure of identically zero depth-3 circuits with bounded top fan-in.

Interpolation of Depth-3 Arithmetic Circuits with Two Multiplication Gates

In this paper we consider the problem of constructing a small arithmetic circuit for a polynomial for which we have oracle access. Our focus is on $n$-variate polynomials, over a finite field $\mathbb{F}$, that have depth-3 arithmetic circuits (with an addition gate at the top) with two multiplication gates of degree at most $d$. We obtain the following results: 1. Multilinear case. When the circuit is multilinear (multiplication gates compute multilinear polynomials) we give an algorithm that outputs, with probability $1-o(1)$, all the depth-3 circuits with two multiplication gates computing the polynomial. The running time of the algorithm is $\operatorname{poly}(n,|\mathbb{F}|)$. 2. General case. When the circuit is not multilinear we give a quasi-polynomial (in $n,d,|\mathbb{F}|$) time algorithm that outputs, with probability $1-o(1)$, a succinct representation of the polynomial. In particular, if the depth-3 circuit for the polynomial is not of small depth-3 rank (namely, after removing the g.c.d. (greatest common divisor) of the two multiplication gates, the remaining linear functions span a not too small linear space), then we output the depth-3 circuit itself. In the case that the rank is small we output a depth-3 circuit with a quasi-polynomial number of multiplication gates. $\diamond$ Prior to our work there have been several interpolation algorithms for restricted models. However, all the techniques used there completely fail when dealing with depth-3 circuits with even just two multiplication gates. Our proof technique is new and relies on the factorization algorithm for multivariate black-box polynomials, on lower bounds on the length of linear locally decodable codes with two queries, and on a theorem regarding the structure of identically zero depth-3 circuits with four multiplication gates.

Cryptographic hardness for learning intersections of halfspaces

We give the first representation-independent hardness results for PAC learning intersections of halfspaces, a central concept class in computational learning theory. Our hardness results are derived from two public-key cryptosystems due to Regev, which are based on the worst-case hardness of well-studied lattice problems. Specifically, we prove that a polynomial-time algorithm for PAC learning intersections of n ϵ halfspaces (for a constant ϵ > 0 ) in n dimensions would yield a polynomial-time solution to O ˜ ( n 1.5 ) - uSVP (unique shortest vector problem). We also prove that PAC learning intersections of n ϵ low-weight halfspaces would yield a polynomial-time quantum solution to O ˜ ( n 1.5 ) - SVP and O ˜ ( n 1.5 ) - SIVP (shortest vector problem and shortest independent vector problem, respectively). Our approach also yields the first representation-independent hardness results for learning polynomial-size depth-2 neural networks and polynomial-size depth-3 arithmetic circuits.

Computing Elementary Symmetric Polynomials with a Subpolynomial Numberof Multiplications

Elementary symmetric polynomials $S_n^k$ are the building blocks of symmetric polynomials. In this work we prove that for constant $k$'s, $S_n^k$ modulo composite numbers m=p1p2 can be computed using only no(1) multiplications if the coefficients of monomials xi1xi2. . . xik are allowed to be 1 either mod p1 or mod p2 but not necessarily both. To the best of our knowledge, no previous result yielded even a sublinear (i.e., $n^{\varepsilon}$, $0<\varepsilon<1$) number of multiplications for similar tasks. Moreover, our algorithm fits in the model of the most restrictive depth-3 arithmetic circuits (homogeneous, multilinear, or the graph model). In contrast, by a lower bound of Nisan and Wigderson [Comput. Complexity, 6 (1997), pp. 217--234], any homogeneous depth-3 circuit needs size $\Omega((n/2k)^{k/2})$ for computing $S_n^k$ modulo primes. Moreover, the number of multiplications in our algorithm remains sublinear while k=O(log log n). Our results generalize for other nonprime-power composite moduli as well. The proof uses perfect hashing functions and the famous BBR polynomial of Barrington, Beigel, and Rudich.

Depth-3 arithmetic circuits over fields of characteristic zero

In this paper we prove quadratic lower bounds for depth-3 arithmetic circuits over fields of characteristic zero. Such bounds are obtained for the elementary symmetric functions, the (trace of) iterated matrix multiplication, and the determinant. As corollaries we get the first nontrivial lower bounds for computing polynomials of constant degree, and a gap between the power of depth-3 arithmetic circuits and depth-4 arithmetic circuits. We also give new shorter formulae of constant depth for the elementary symmetric functions.¶The main technical contribution relates the complexity of computing a polynomial in this model to the wealth of partial derivatives it has on every affine subspace of small co-dimension. Lower bounds for related models utilize an algebraic analog of the Neciporuk lower bound on Boolean formulae.

Some Results on Uniform Arithmetic Circuit Complexity

We introduce a natural set of arithmetic expressions and define the complexity class AE to consist of all those arithmetic functions (over the fields F_(2)n) that are described by these expressions. We show that AE coincides with the class of functions that are computable with constant depth and polynomial size unbounded fan-in arithmetic circuits satisfying a natural uniformity constraint (DLOGTIME-uniformity). A 1-input and 1-output arithmetic function over the fields F_(2)n may be identified with an &lt;em&gt;n&lt;/em&gt;-input and an n-output Boolean function when field elements are represented as bit strings. We prove that if some such representation is X-uniform (where X is P or DLOGTIME) then the arithmetic complexity of a function (measured with X-uniform unbounded fan-in arithmetic circuits) is identical to the Boolean complexity of this function (measured with X-uniform threshold circuits). We show the existence of a P-uniform representation and we give partial results concerning the existence of representations with more restrictive uniformity properties.