Polynomial Threshold Functions Research Articles

Combined weight and density bounds on the polynomial threshold function representation of Boolean functions

In an earlier report it was shown that an arbitrary n-variable Boolean function f can be represented as a polynomial threshold function (PTF) with 0.75×2n or less number of monomials. In this report, we derive an upper bound on the absolute value of the (integer) weights of a PTF that represents f and still obeys the aforementioned density bound. To our knowledge this provides the best combined bound on the PTF density (number of monomials) and PTF weight (sum of the coefficient magnitudes) of general Boolean functions. For the special case of bent functions, it is found that any n-variable bent function can be represented with integer coefficients less than or equal to 2n with density no more than 0.75×2n, and for the case of m-sparse Boolean functions that are almost constant except for small (m≪2n) number of variable assignments, it is shown that they can be represented with small weight PTFs with density at most m+2n−1. In addition, tight PTF weight bounds with conformance to the density bound of 0.75×2n are numerically obtained for the general Boolean functions up to 6 variables.

A #SAT Algorithm for Small Constant-Depth Circuits with PTF gates

We show that there is a better-than-brute-force algorithm that, when given a small constant-depth Boolean circuit C made up of gates that compute constant-degree Polynomial Threshold functions or PTFs (i.e., Boolean functions that compute signs of constant-degree polynomials), counts the number of satisfying assignments to C in significantly better than brute-force time. Formally, for any constants d, k, there is an \(\varepsilon > 0\) such that the zero-error randomized algorithm counts the number of satisfying assignments to a given depth-d circuit C made up of k-PTF gates such that C has at most \(n^{1+\varepsilon }\) many wires. The algorithm runs in time \(2^{n-n^{\Omega (\varepsilon )}}.\) Before our result, no algorithm for beating brute-force search was known for counting the number of satisfying assignments even for a single degree-k PTF (which is a depth-1 circuit with linearly many wires).We give two different algorithms for the case of a single PTF. The first uses a learning algorithm for learning degree-1 PTFs (or Linear Threshold Functions) using comparison queries due to Kane, Lovett and Moran (STOC 2018), and the second uses a proof of Hofmeister (COCOON 1996) for converting a degree-1 PTF to a depth-two threshold circuit with small weights. We show that both these ideas fit nicely into a memoization approach that yields the #SAT algorithms.

Algorithms and Lower Bounds for De Morgan Formulas of Low-Communication Leaf Gates

The class FORMULA[s]∘G consists of Boolean functions computable by size- s De Morgan formulas whose leaves are any Boolean functions from a class G. We give lower bounds and (SAT, Learning, and pseudorandom generators ( PRG s )) algorithms for FORMULA[n 1.99 ]∘G, for classes G of functions with low communication complexity . Let R (k) G be the maximum k -party number-on-forehead randomized communication complexity of a function in G. Among other results, we show the following: • The Generalized Inner Product function GIP k n cannot be computed in FORMULA[s]° G on more than 1/2+ε fraction of inputs for s=o(n 2 /k⋅4 k ⋅R (k) (G)⋅log⁡(n/ε)⋅log⁡(1/ε)) 2 ). This significantly extends the lower bounds against bipartite formulas obtained by [62]. As a corollary, we get an average-case lower bound for GIP k n against FORMULA[n 1.99 ]∘PTF k −1 , i.e., sub-quadratic-size De Morgan formulas with degree-k-1) PTF ( polynomial threshold function ) gates at the bottom. Previously, it was open whether a super-linear lower bound holds for AND of PTFs. • There is a PRG of seed length n/2+O(s⋅R (2) (G)⋅log⁡(s/ε)⋅log⁡(1/ε)) that ε-fools FORMULA[s]∘G. For the special case of FORMULA[s]∘LTF, i.e., size- s formulas with LTF ( linear threshold function ) gates at the bottom, we get the better seed length O(n 1/2 ⋅s 1/4 ⋅log⁡(n)⋅log⁡(n/ε)). In particular, this provides the first non-trivial PRG (with seed length o(n)) for intersections of n halfspaces in the regime where ε≤1/n, complementing a recent result of [45]. • There exists a randomized 2 n-t #SAT algorithm for FORMULA[s]∘G, where t=Ω(n\√s⋅log 2 ⁡(s)⋅R (2) (G))/1/2. In particular, this implies a nontrivial #SAT algorithm for FORMULA[n 1.99 ]∘LTF. • The Minimum Circuit Size Problem is not in FORMULA[n 1.99 ]∘XOR; thereby making progress on hardness magnification, in connection with results from [14, 46]. On the algorithmic side, we show that the concept class FORMULA[n 1.99 ]∘XOR can be PAC-learned in time 2 O(n/log n) .

Coded Computing for Secure Boolean Computations

The growing size of modern datasets necessitates splitting a large scale computation into smaller computations and operate in a distributed manner. Adversaries in a distributed system deliberately send erroneous data in order to affect the computation for their benefit. Boolean functions are the key components of many applications, e.g., verification functions in blockchain systems and design of cryptographic algorithms. We consider the problem of computing a Boolean function in a distributed computing system with particular focus on security against Byzantine workers. Any Boolean function can be modeled as a multivariate polynomial with high degree in general. However, the security threshold (i.e., the maximum number of adversarial workers can be tolerated such that the correct results can be obtained) provided by the recent proposed Lagrange Coded Computing (LCC) can be extremely low if the degree of the polynomial is high. We propose three different schemes called coded Algebraic normal form (ANF), coded Disjunctive normal form (DNF) and coded polynomial threshold function (PTF). The key idea of the proposed schemes is to model it as the concatenation of some low-degree polynomials and threshold functions. In terms of the security threshold, we show that the proposed coded ANF and coded DNF are optimal by providing a matching outer bound.

On XOR lemmas for the weight of polynomial threshold functions

A multilinear polynomial p is said to sign-represent a Boolean function f:{−1,1}n→{−1,1} if f(x)=sgn(p(x)) for all x∈{−1,1}n. In this paper, we consider the length and weight of polynomials sign-representing Boolean functions of the form ⊕kf, the XOR of k copies of f on disjoint sets of variables.Firstly, we show that for an infinite family of functions f, a naive construction does not yield a shortest polynomial sign-representing ⊕kf. More precisely, we give a construction of polynomials sign-representing ⊕kANDn whose length is strictly smaller than the k-th power of the minimum length of a polynomial sign-representing ANDn, for every k≥2 and n≥2 except for k=n=2. Previously, such polynomials were known only for n=2 (Sezener and Oztop, 2015). A similar result for the weight is also provided. Secondly, we introduce a parameter vf⁎ of a Boolean function f and show that the k-th root of the minimum weight of a polynomial sign-representing ⊕kf converges between vf⁎ and (vf⁎)2 as k goes to infinity.

Polynomial Threshold Functions, Hyperplane Arrangements, and Random Tensors

A simple way to generate a Boolean function is to take the sign of a real polynomial in $n$ variables. Such Boolean functions are called polynomial threshold functions. How many low-degree polynomi...

On the Representational Power of Restricted Boltzmann Machines for Symmetric Functions and Boolean Functions.

Restricted Boltzmann machines (RBMs) are used to build deep-belief networks that are widely thought to be one of the first effective deep learning neural networks. This paper studies the ability of RBMs to represent distributions over {0,1}n via softplus/hardplus RBM networks. It is shown that any distribution whose density depends on the number of 1's in their input can be approximated with arbitrarily high accuracy by an RBM of size 2n+1 , which improves the result of a previous study by reducing the size from n2 to 2n+1 . A theorem for representing partially symmetric Boolean functions by softplus RBM networks is established. Accordingly, the representational power of RBMs for distributions whose mass represents the Boolean functions is investigated in comparison with that of threshold circuits and polynomial threshold functions. It is shown that a distribution over {0,1}n whose mass represents a Boolean function can be computed with a given margin δ by an RBM of size and parameters bounded by polynomials in n , if and only if it can be computed by a depth-2 threshold circuit with size and parameters bounded by polynomials in n .

A new central limit theorem and decomposition for Gaussian polynomials, with an application to deterministic approximate counting

One of the main results of this paper is a new multidimensional central limit theorem (CLT) for multivariate polynomials under Gaussian inputs. Roughly speaking, the new CLT shows that any collection of Gaussian polynomials with small eigenvalues (suitably defined) must have a joint distribution which is close to a multidimensional Gaussian distribution. The new CLT is proved using tools from Malliavin calculus and Stein’s method. A second main result of the paper, which complements the new CLT, is a new decomposition theorem for low-degree multilinear polynomials over Gaussian inputs. Roughly speaking, this result shows that (up to some small error) any such polynomial is very close to a polynomial which can be decomposed into a bounded number of multilinear polynomials all of which have extremely small eigenvalues. An important feature of this decomposition theorem is the delicate control obtained between the number of polynomials in the decomposition versus their eigenvalues. As the main application of these results, we give a deterministic algorithm for approximately counting satisfying assignments of a degree-d polynomial threshold function (PTF) over the domain $$\{-1,1\}^n$$ ; this is a well-studied problem from theoretical computer science. More precisely, given as input a degree-d polynomial $$p(x_1,\dots ,x_n)$$ over $${{\mathbb {R}}}^n$$ and a parameter $$\epsilon > 0$$ , the algorithm approximates $$\begin{aligned} \mathop {\mathbf{Pr}}_{x \sim \{-1,1\}^n}[p(x) \ge 0] \end{aligned}$$ to within an additive $$\pm \epsilon $$ in time $$O_{d,\epsilon }(1)\cdot \mathrm {poly}(n^d)$$ . (Since it is NP-hard to determine whether the above probability is nonzero, any sort of efficient multiplicative approximation is almost certainly impossible even for randomized algorithms.) Note that the running time of the algorithm (as a function of $$n^d$$ , the number of coefficients of a degree-d PTF) is a fixed polynomial. The fastest previous algorithm for this problem (Kane, CoRR, arXiv:1210.1280 , 2012), based on constructions of unconditional pseudorandom generators for degree-d PTFs, runs in time $$n^{O_{d,c}(1) \cdot \epsilon ^{-c}}$$ for all $$c > 0$$ .

Corrigendum to “Supervised and Unsupervised Subband Adaptive Denoising Frameworks with Polynomial Threshold Function”

Corrigendum to “Supervised and Unsupervised Subband Adaptive Denoising Frameworks with Polynomial Threshold Function”

A structure theorem for poorly anticoncentrated polynomials of Gaussians and applications to the study of polynomial threshold functions

We prove a structural result for degree-$d$ polynomials. In particular, we show that any degree-$d$ polynomial, $p$ can be approximated by another polynomial, $p_{0}$, which can be decomposed as some function of polynomials $q_{1},\ldots,q_{m}$ with $q_{i}$ normalized and $m=O_{d}(1)$, so that if $X$ is a Gaussian random variable, the probability distribution on $(q_{1}(X),\ldots,q_{m}(X))$ does not have too much mass in any small box. Using this result, we prove improved versions of a number of results about polynomial threshold functions, including producing better pseudorandom generators, obtaining a better invariance principle, and proving improved bounds on noise sensitivity.

Supervised and Unsupervised Subband Adaptive Denoising Frameworks with Polynomial Threshold Function

Unlike inflexible structure of soft and hard threshold function, a unified linear matrix form with flexible structure for threshold function is proposed. Based on the unified linear flexible structure threshold function, both supervised and unsupervised subband adaptive denoising frameworks are established. To determine flexible coefficients, a direct mean-square error (MSE) minimization is conducted in supervised denoising while Stein’s unbiased risk estimate as a MSE estimate is minimized in unsupervised denoising. The SURE rule requires no hypotheses or a priori knowledge about clean signals. Furthermore, we discuss conditions to obtain optimal coefficients for both supervised and unsupervised subband adaptive denoising frameworks. Applying an Odd-Term Reserving Polynomial (OTRP) function as concrete threshold function, simulations for polynomial order, denoising performance, and noise effect are conducted. Proper polynomial order and noise effect are analyzed. Both proposed methods are compared with soft and hard based denoising technologies—VisuShrink, SureShrink, MiniMaxShrink, and BayesShrink—in denoising performance simulation. Results show that the proposed approaches perform better in both MSE and signal-to-noise ratio (SNR) sense.

Upper bounds on Fourier entropy

Given a function f:{0,1}n→{+1,−1}, its Fourier Entropy is defined to be −∑Sfˆ(S)2log⁡fˆ(S)2, where fˆ denotes the Fourier transform of f. In the analysis of Boolean functions, an outstanding open question is a conjecture of Friedgut and Kalai, 1996 [3], called the Fourier Entropy Influence (FEI) Conjecture, asserting that the Fourier Entropy of any Boolean function f is bounded above, up to a constant factor, by the total influence (=average sensitivity) of f.In this paper we give several upper bounds on the Fourier Entropy. We first give upper bounds on the Fourier Entropy of Boolean functions in terms of several complexity measures that are known to be bigger than the influence. These complexity measures include, among others, the logarithm of the number of leaves and the average depth of a parity decision tree. We then show that for the class of Linear Threshold Functions (LTF), the Fourier Entropy is O(n). It is known that the average sensitivity for the class of LTF is Θ(n). We also establish a bound of Od(n1−14d+6) for general degree-d polynomial threshold functions. Our proof is based on a new upper bound on the derivative of noise sensitivity. Next we proceed to show that the FEI Conjecture holds for read-once formulas that use AND, OR, XOR, and NOT gates. The last result is independent of a result due to O'Donnell and Tan [1] for read-once formulas with arbitrary gates of bounded fan-in, but our proof is completely elementary and very different from theirs. Finally, we give a general bound involving the first and second moments of sensitivities of a function (average sensitivity being the first moment), which holds for real-valued functions as well.

Approximating Boolean Functions with Depth-2 Circuits

We study the complexity of approximating Boolean functions with disjunctive normal forms (DNFs) and other depth-2 circuits, exploring two main directions: universal bounds on the approximability of all Boolean functions, and the approximability of the parity function. In the first direction, our main positive results are the first nontrivial universal upper bounds on approximability by DNFs: (a) every Boolean function can be $\epsilon$-approximated by a DNF of size $O_\epsilon(2^n/\log n)$, and (b) every Boolean function can be $\epsilon$-approximated by a DNF of width $c_\epsilon\, n$, where $c_\epsilon < 1$. Our techniques extend broadly to give strong universal upper bounds on approximability by various depth-2 circuits that generalize DNFs, including the intersection of halfspaces, low-degree Polynomial threshold functions, and unate functions. We show that the parameters of our constructions almost match the information-theoretic inapproximability of a random function. In the second direction our mai...

The communication complexity of addition

Suppose each of k≤no(1) players holds an n-bit number xi in its hand. The players wish to determine if ?i≤kxi=s. We give a public-coin protocol with error 1% and communication O(k logk). The communication bound is independent of n, and for k?3 improves on the O(k logn) bound by Nisan (Bolyai Soc. Math. Studies; 1993). Our protocol also applies to addition modulo m. In this case we give a matching (public-coin) Ω(k logk) lower bound for various m. We also obtain some lower bounds over the integers, including Ω (k log logk) for protocols that are one-way, like ours. We give a protocol to determine if ?xi >s with error 1% and communication O(k logk) log n. For k?3 this improves on Nisan's O(k log2n) bound. A similar improvement holds for computing degree-(k?1) polynomial-threshold functions in the number-on-forehead model. We give a (public-coin, 2-player, tight) Ω(logn) lower bound to determine if x1 >x2. This improves on the Ω(?logn) bound by Smirnov (1988). Troy Lee informed us in January 2013 that an Ω(logn) lower bound may also be obtained by combining a result in learning theory by Forster et al. (2003) with a result by Linial and Shraibman (2009). As an application, we show that polynomial-size AC0 circuits augmented with O(1) threshold (or symmetric) gates cannot compute cryptographic pseudorandom functions, extending the result about AC0 by Linial, Mansour, and Nisan (J. ACM; 1993).

Polynomial threshold functions and Boolean threshold circuits

We initiate a comprehensive study of the complexity of computing Boolean functions by polynomial threshold functions (PTFs) on general Boolean domains (as opposed to domains such as {0,1}n or {−1,1}n that enforce multilinearity). A typical example of such a general Boolean domain, for the purpose of our results, is {1,2}n. We are mainly interested in the length (the number of monomials) of PTFs, with their degree and weight being of secondary interest.First we motivate the study of PTFs over the {1,2}n domain by showing their close relation to depth two threshold circuits. In particular we show that PTFs of polynomial length and polynomial degree compute exactly the functions computed by polynomial size THR∘MAJ circuits. We note that known lower bounds for THR∘MAJ circuits extend to the likely strictly stronger model of PTFs (with no degree restriction). We also show that a “max-plus” version of PTFs is related to AC0∘THR circuits.We exploit this connection to gain a better understanding of threshold circuits. In particular, we show that (super-logarithmic) lower bounds for 3-player randomized communication protocols with unbounded error would yield (super-polynomial) size lower bounds for THR∘THR circuits.Finally, having thus motivated the model, we initiate structural studies of PTFs. These include relationships between weight and degree of PTFs, and a degree lower bound for PTFs of constant length.

The correct exponent for the Gotsman–Linial Conjecture

We prove new bounds on the average sensitivity of polynomial threshold functions. In particular, we show that for f, a degree-d polynomial threshold function in n variables that $$\mathbb{AS}(f) \leq \sqrt{n}(\log(n))^{O(d log(d))}2^{O(d^2 log(d))}.$$ AS ( f ) ≤ n ( log ( n ) ) O ( d l o g ( d ) ) 2 O ( d 2 l o g ( d ) ) . This bound amounts to a significant improvement over previous bounds, and in particular, for fixed d gives the same asymptotic exponent of n as the one predicted by the Gotsman---Linial Conjecture.

Average Sensitivity and Noise Sensitivity of Polynomial Threshold Functions

We give the first nontrivial upper bounds on the Boolean average sensitivity and noise sensitivity of degree-$d$ polynomial threshold functions (PTFs). Our bound on the Boolean average sensitivity of PTFs represents the first progress toward the resolution of a conjecture of Gotsman and Linial [Combinatorica, 14 (1994), pp. 35--50], which states that the symmetric function slicing the middle $d$ layers of the Boolean hypercube has the highest average sensitivity of all degree-$d$ PTFs. Via the $L_1$ polynomial regression algorithm of Kalai et al. [SIAM J. Comput., 37 (2008), pp. 1777--1805], our bound on Boolean noise sensitivity yields the first polynomial-time agnostic learning algorithm for the broad class of constant-degree PTFs under the uniform distribution. To obtain our bound on the Boolean average sensitivity of PTFs, we generalize the “critical-index” machinery of [R. Servedio, Comput. Complexity, 16 (2007), pp. 180--209] (which in that work applies to halfspaces, i.e., degree-1 PTFs) to general...

A Regularity Lemma and Low-Weight Approximators for Low-Degree Polynomial Threshold Functions

We give a lemma for degree-d polynomial threshold functions (PTFs) over the Boolean cube {−1,1}^n. Roughly speaking, this result shows that every degree-d PTF can be decomposed into a constant number of subfunctions such that almost all of the subfunctions are close to being PTFs. Here a regular PTF is a PTF sign(p(x)) where the influence of each variable on the polynomial p(x) is a small fraction of the total influence of p. As an application of this regularity lemma, we prove that for any constants d >= 1, eps > 0, every degree-d PTF over n variables can be approximated to accuracy eps by a constant degree PTF that has integer weights of total magnitude O(n^d). This weight bound is shown to be optimal up to logarithmic factors.

Deterministic Approximate Counting for Degree-2 Polynomial Threshold Functions.

We give a deterministic algorithm for approximately computing the fraction of Boolean assignments that satisfy a degree-2 polynomial threshold function. Given a degree-2 input polynomial p(x1, . . . , xn) and a parameter > 0, the algorithm approximates Prx∼{−1,1}n [p(x) ≥ 0] to within an additive ± in time poly(n, 2 ). Note that it is NP-hard to determine whether the above probability is nonzero, so any sort of multiplicative approximation is almost certainly impossible even for efficient randomized algorithms. This is the first deterministic algorithm for this counting problem in which the running time is polynomial in n for = o(1). For “regular” polynomials p (those in which no individual variable’s influence is large compared to the sum of all n variable influences) our algorithm runs in poly(n, 1/ ) time. The algorithm also runs in poly(n, 1/ ) time to approximate Prx∼N(0,1)n [p(x) ≥ 0] to within an additive ± , for any degree-2 polynomial p. As an application of our counting result, we give a deterministic FPT multiplicative (1± )-approximation algorithm to approximate the k-th absolute moment Ex∼{−1,1}n [|p(x)|] of a degree-2 polynomial. The algorithm’s running time is of the form poly(n) · f(k, 1/ ). ∗anindya@math.ias.edu. Research supported by Templeton Foundation Grant 21674 and NSF CCF-1149843. †ilias.d@ed.ac.uk. Some of this work was done while the author was at UC Berkeley supported by a Simons Fellowship. ‡rocco@cs.columbia.edu. Supported by NSF grants CNS-0716245, CCF-0915929, and CCF-1115703. ISSN 1433-8092 Electronic Colloquium on Computational Complexity, Report No. 172 (2013)

Pseudorandom Generators for Polynomial Threshold Functions

We study the natural question of constructing pseudorandom generators (PRGs) for low-degree polynomial threshold functions (PTFs). We give a PRG with seed length $\log n/\epsilon^{O(d)}$ fooling degree $d$ PTFs with error at most $\epsilon$. Previously, no nontrivial constructions were known even for quadratic threshold functions and constant error $\epsilon$. For the class of degree $1$ threshold functions or halfspaces, previously only PRGs with seed length $O(\log n \log^2(1/\epsilon)/\epsilon^2)$ were known. We improve this dependence on the error parameter and construct PRGs with seed length $O(\log n + \log^2 (1/\epsilon))$ that $\epsilon$-fool halfspaces. We also obtain PRGs with similar seed lengths for fooling halfspaces over the $n$-dimensional unit sphere. The main theme of our constructions and analysis is the use of invariance principles to construct PRGs. We also introduce the notion of monotone read-once branching programs, which is key to improving the dependence on the error rate $\epsilo...