DNF Formulas Research Articles

The Sign-Rank of AC$^0$

The sign-rank of a matrix $A=[A_{ij}]$ with $\pm1$ entries is the least rank of a real matrix $B=[B_{ij}]$ with $A_{ij}B_{ij}>0$ for all $i,j$. We obtain the first exponential lower bound on the sign-rank of a function in $\mathsf{AC}^0$. Namely, let $f(x,y)=\bigwedge_{i=1,\dots,m}\bigvee_{j=1,\dots,m^2}(x_{ij}\wedge y_{ij})$. We show that the matrix $[f(x,y)]_{x,y}$ has sign-rank $\exp(\Omega(m))$. This in particular implies that $\Sigma_2^{cc}\not\subseteq\mathsf{UPP}^{cc}$, which solves a longstanding open problem in communication complexity posed by Babai, Frankl, and Simon [Proceedings of the 27th Symposium on Foundations of Computer Science (FOCS), 1986, pp. 337-347]. Our result additionally implies a lower bound in learning theory. Specifically, let $\phi_1,\dots,\phi_r:\{0,1\}^n\to\mathbb{R}$ be functions such that every DNF formula $f:\{0,1\}^n\to\{-1,+1\}$ of polynomial size has the representation $f\equiv\mathrm{sgn}(a_1\phi_1+\dots+a_r\phi_r)$ for some reals $a_1,\dots,a_r$. We prove that then $r\geqslant\exp(\Omega(n^{1/3}))$, which essentially matches an upper bound of $\exp(\tilde{O}(n^{1/3}))$, due to Klivans and Servedio [J. Comput. System Sci., 68 (2004), pp. 303-318]. Finally, our work yields the first exponential lower bound on the size of threshold-of-majority circuits computing a function in $\mathsf{AC}^0$. This substantially generalizes and strengthens the results of Krause and Pudlak [Theoret. Comput. Sci., 174 (1997), pp. 137-156].

A Simple Proof of Bazzi’s Theorem

Linial and Nisan [1990] asked if any polylog-wise independent distribution fools any function in AC 0 . In a recent remarkable development, Bazzi solved this problem for the case of DNF formulas. The aim of this note is to present a simplified version of his proof.

Pattern minimization problems over recursive data types

In the context of program verification in an interactive theorem prover, we study the problem of transforming function definitions with ML-style (possibly overlapping) pattern matching into minimal sets of independent equations. Since independent equations are valid unconditionally, they are better suited for the equational proof style using induction and rewriting, which is often found in proofs in theorem provers or on paper. We relate the problem to the well-known minimization problem for propositional DNF formulas and show that it is £ P /2-complete. We then develop a concrete algorithm to compute minimal patterns, which naturally generalizes the standard Quine-McCluskey procedure to the domain of term patterns.

Hardness of approximate two-level logic minimization and PAC learning with membership queries

Producing a small DNF expression consistent with given data is a classical problem in computer science that occurs in a number of forms and has numerous applications. We consider two standard variants of this problem. The first one is two-level logic minimization or finding a minimum DNF formula consistent with a given complete truth table (TT-MinDNF). This problem was formulated by Quine in 1952 and has been since one of the key problems in logic design. It was proved NP -complete by Masek in 1979. The best known polynomial approximation algorithm is based on a reduction to the SET-COVER problem and produces a DNF formula of size O ( d ⋅ OPT ) , where d is the number of variables. We prove that TT-MinDNF is NP -hard to approximate within d γ for some constant γ > 0 , establishing the first inapproximability result for the problem. The other DNF minimization problem we consider is PAC learning of DNF expressions when the learning algorithm must output a DNF expression as its hypothesis (referred to as proper learning). We prove that DNF expressions are NP -hard to PAC learn properly even when the learner has access to membership queries, thereby answering a long-standing open question due to Valiant [L.G. Valiant, A theory of the learnable, Comm. ACM 27 (11) (1984) 1134–1142]. Finally, we provide a concrete connection between these variants of DNF minimization problem. Specifically, we prove that inapproximability of TT-MinDNF implies hardness results for restricted proper learning of DNF expressions with membership queries even when learning with respect to the uniform distribution only.

Polylogarithmic independence fools AC 0 circuits

We prove that poly-sized AC 0 circuits cannot distinguish a polylogarithmically independent distribution from the uniform one. This settles the 1990 conjecture by Linial and Nisan [1990]. The only prior progress on the problem was by Bazzi [2007], who showed that O (log 2 n )-independent distributions fool poly-size DNF formulas. [Razborov 2008] has later given a much simpler proof for Bazzi's theorem.

An improvement on the complexity of factoring read-once Boolean functions

Read-once functions have gained recent, renewed interest in the fields of theory and algorithms of Boolean functions, computational learning theory and logic design and verification. In an earlier paper [M.C. Golumbic, A. Mintz, U. Rotics, Factoring and recognition of read-once functions using cographs and normality, and the readability of functions associated with partial k-trees, Discrete Appl. Math. 154 (2006) 1465–1677], we presented the first polynomial-time algorithm for recognizing and factoring read-once functions, based on a classical characterization theorem of Gurvich which states that a positive Boolean function is read-once if and only if it is normal and its co-occurrence graph is P 4 -free. In this note, we improve the complexity bound by showing that the method can be modified slightly, with two crucial observations, to obtain an O ( n | f | ) implementation, where | f | denotes the length of the DNF expression of a positive Boolean function f, and n is the number of variables in f. The previously stated bound was O ( n 2 k ) , where k is the number of prime implicants of the function. In both cases, f is assumed to be given as a DNF formula consisting entirely of the prime implicants of the function.

Projective DNF formulae and their revision

Combinatorial and learnability results are proven for projective disjunctive normal forms, a class of DNF expressions introduced by Valiant.

The complexity of properly learning simple concept classes

We consider the complexity of properly learning concept classes, i.e. when the learner must output a hypothesis of the same form as the unknown concept. We present the following new upper and lower bounds on well-known concept classes: • We show that unless NP = RP , there is no polynomial-time PAC learning algorithm for DNF formulas where the hypothesis is an OR-of-thresholds. Note that as special cases, we show that neither DNF nor OR-of-thresholds are properly learnable unless NP = RP . Previous hardness results have required strong restrictions on the size of the output DNF formula. We also prove that it is NP -hard to learn the intersection of ℓ ⩾ 2 halfspaces by the intersection of k halfspaces for any constant k ⩾ 0 . Previous work held for the case when k = ℓ . • Assuming that NP ⊈ DTIME ( 2 n ϵ ) for a certain constant ϵ < 1 we show that it is not possible to learn size s decision trees by size s k decision trees for any k ⩾ 0 . Previous hardness results for learning decision trees held for k ⩽ 2 . • We present the first non-trivial upper bounds on properly learning DNF formulas. More specifically, we show how to learn size s DNF by DNF in time 2 O ˜ ( n log s ) . The hardness results for DNF formulas and intersections of halfspaces are obtained via specialized graph products for amplifying the hardness of approximating the chromatic number as well as applying recent work on the hardness of approximate hypergraph coloring. The hardness results for decision trees, as well as the new upper bounds, are obtained by developing a connection between automatizability in proof complexity and learnability, which may have other applications.

DNF are teachable in the average case

We study the average number of well-chosen labeled examples that are required for a helpful teacher to uniquely specify a target function within a concept class. This “average teaching dimension” has been studied in learning theory and combinatorics and is an attractive alternative to the “worst-case” teaching dimension of Goldman and Kearns which is exponential for many interesting concept classes. Recently Balbach showed that the classes of 1-decision lists and 2-term DNF each have linear average teaching dimension. As our main result, we extend Balbach’s teaching result for 2-term DNF by showing that for any 1≤s≤2Θ(n), the well-studied concept classes of at-most-s-term DNF and at-most-s-term monotone DNF each have average teaching dimension O(ns). The proofs use detailed analyses of the combinatorial structure of “most” DNF formulas and monotone DNF formulas. We also establish asymptotic separations between the worst-case and average teaching dimension for various other interesting Boolean concept classes such as juntas and sparse GF 2 polynomials.

A general dimension for query learning

We introduce a combinatorial dimension that characterizes the number of queries needed to exactly (or approximately) learn concept classes in various models. Our general dimension provides tight upper and lower bounds on the query complexity for all sorts of queries, not only for example-based queries as in previous works. As an application we show that for learning DNF formulas, unspecified attribute value membership and equivalence queries are not more powerful than standard membership and equivalence queries. Further, in the approximate learning setting, we use the general dimension to characterize the query complexity in the statistical query as well as the learning by distances model. Moreover, we derive close bounds on the number of statistical queries needed to approximately learn DNF formulas.

On Learning Random DNF Formulas Under the Uniform Distribution

We study the average-case learnability of DNF formulas in the model of learning from uniformly distributed random examples. We define a natural model of random monotone DNF formulas and give an efficient algorithm which with high probability can learn, for any fixed constant γ>0, a random t-term monotone DNF for any t = O(n2−γ). We also define a model of random nonmonotone DNF and give an efficient algorithm which with high probability can learn a random t-term DNF for any t=O(n3/2−γ). These are the first known algorithms that can successfully learn a broad class of polynomial-size DNF in a reasonable average-case model of learning from random examples.

The complexity of learning concept classes with polynomial general dimension

The general dimension is a combinatorial measure that characterizes the number of queries needed to learn a concept class. We use this notion to show that any p-evaluatable concept class with polynomial query complexity can be learned in polynomial time with the help of an oracle in the polynomial hierarchy, where the complexity of the required oracle depends on the query-types used by the learning algorithm. In particular, we show that for subset and superset queries an oracle in Σ 3 P suffices. Since the concept class of DNF formulas has polynomial query complexity with respect to subset and superset queries with DNF formulas as hypotheses, it follows that DNF formulas are properly learnable in polynomial time with subset and superset queries and the help of an oracle in Σ 3 P . We also show that the required oracle in our main theorem cannot be replaced by an oracle in a lower level of the polynomial-time hierarchy, unless the hierarchy collapses.

Prediction-hardness of acyclic conjunctive queries

A conjunctive query problem is a problem to determine whether or not a tuple belongs to the answer of a conjunctive query over a database. In this paper, a tuple, a conjunctive query and a database in relational database theory are regarded as a ground atom, a nonrecursive function-free definite clause and a finite set of ground atoms, respectively, in inductive logic programming terminology. An acyclic conjunctive query problem is a conjunctive query problem with acyclicity. Concerned with the acyclic conjunctive query problem, in this paper, we present the hardness results of predicting acyclic conjunctive queries from an instance with a j- database of which predicate symbol is at most j-ary. Also we deal with two kinds of instances, a simple instance as a set of ground atoms and an extended instance as a set of pairs of a ground atom and a description. We mainly show that, from both a simple and an extended instances, acyclic conjunctive queries are not polynomial-time predictable with j-databases ( j ⩾ 3 ) under the cryptographic assumptions, and predicting acyclic conjunctive queries with 2-databases is as hard as predicting DNF formulas. Hence, the acyclic conjunctive queries become a natural example that the equivalence between subsumption-efficiency and efficient pac-learnability from both a simple and an extended instances collapses.

Exact learning of DNF formulas using DNF hypotheses

We show the following: (a) For any ε > 0 , log ( 3 + ε ) n -term DNF cannot be polynomial-query learned with membership and strongly proper equivalence queries. (b) For sufficiently large t, t-term DNF formulas cannot be polynomial-query learned with membership and equivalence queries that use t 1 + ε -term DNF formulas as hypotheses, for some ε < 1 (c) Read-thrice DNF formulas are not polynomial-query learnable with membership and proper equivalence queries. (d) log n -term DNF formulas can be polynomial-query learned with membership and proper equivalence queries. (This complements a result of Bshouty, Goldman, Hancock, and Matar that log n -term DNF can be so learned in polynomial time.) Versions of (a)–(c) were known previously, but the previous versions applied to polynomial- time learning and used complexity theoretic assumptions. In contrast, (a)–(c) apply to polynomial- query learning, imply the results for polynomial-time learning, and do not use any complexity-theoretic assumptions.

Learning DNF from random walks

We consider a model of learning Boolean functions from examples generated by a uniform random walk on { 0 , 1 } n . We give a polynomial time algorithm for learning decision trees and DNF formulas in this model. This is the first efficient algorithm for learning these classes in a natural passive learning model where the learner has no influence over the choice of examples used for learning.

On learning monotone DNF under product distributions

We show that the class of monotone 2 O ( log n ) -term DNF formulae can be PAC learned in polynomial time under the uniform distribution from random examples only. This is an exponential improvement over the best previous polynomial-time algorithms in this model, which could learn monotone o(log 2 n )-term DNF. We also show that various classes of small constant-depth circuits which compute monotone functions are PAC learnable in polynomial time under the uniform distribution. All of our results extend to learning under any constant-bounded product distribution.

On approximating weighted sums with exponentially many terms

Multiplicative weight-update algorithms such as Winnow and Weighted Majority have been studied extensively due to their on-line mistake bounds’ logarithmic dependence on N, the total number of inputs, which allows them to be applied to problems where N is exponential. However, a large N requires techniques to efficiently compute the weighted sums of inputs to these algorithms. In special cases, the weighted sum can be exactly computed efficiently, but for numerous problems such an approach seems infeasible. Thus we explore applications of Markov chain Monte Carlo (MCMC) methods to estimate the total weight. Our methods are very general and applicable to any representation of a learning problem for which the inputs to a linear learning algorithm can be represented as states in a completely connected, untruncated Markov chain. We give theoretical worst-case guarantees on our technique and then apply it to two problems: learning DNF formulas using Winnow, and pruning classifier ensembles using Weighted Majority. We then present empirical results on simulated data indicating that in practice, the time complexity is much better than what is implied by our worst-case theoretical analysis.

The monotone theory for the PAC-model

In this paper we extend the Monotone Theory to the PAC-learning Model with membership queries. Using this extention we show that a DNF formula that has at least one “1/ poly -heavy” clause in one of its CNF representation (a clause that is not satisfied with probability 1/ poly ( n , s ) where n is the number of variables and s is the number of terms in f ) with respect to a distribution D is weakly learnable under this distribution. So DNF that are not weakly learnable under the distribution D do not have any “1/ poly -heavy” clauses in any of their CNF representations. A DNF f is called τ -CDNF if there is τ ′ > τ and a CNF representation of f that contains poly ( n , s ) clauses that τ ′ -approximates f according to a distribution D . We show that the class of all τ -CDNF is weakly ( τ + ϵ )-PAC-learnable with membership queries under the distribution D . We then show how to change our algorithm to a parallel algorithm that runs in polylogarithmic time with a polynomial number of processors. In particular, decision trees are (strongly) PAC-learnable with membership queries under any distribution in parallel in polylogarithmic time with a polynomial number of processors. Finally we show that no efficient parallel exact learning algorithm exists for decision trees.

Learning elementary formal systems with queries

The elementary formal system ( EFS) is a kind of logic programs which directly manipulates strings, and the learnability of the subclass called hereditary EFSs ( HEFSs) has been investigated in the frameworks of the PAC-learning, query-learning, and inductive inference models. The hierarchy of HEFS is expressed by HEFS( m, k, t, r), where m, k, t and r denote the number of clauses, the occurrences of variables in the head, the number of atoms in the body, and the arity of predicate symbols. The present paper deals with the learnability of HEFS in the query learning model using equivalence queries and additional queries such as membership, predicate membership, entailment membership, and dependency queries. We show that the class HEFS(∗,k,t,r) is polynomial-time learnable with the equivalence and predicate membership queries and the class HEFS(∗,k,∗,r) with termination property is polynomial-time learnable with the equivalence, entailment membership, and dependency queries for the unbounded parameter ∗. A lowerbound on the number of queries is presented. We also show that the class HEFS(∗,k,t,r) is hard to learn with the equivalence and membership queries under the cryptographic assumptions. Furthermore, the learnability of the class of unions of regular pattern languages, which is a subclass of HEFSs, is investigated. The bounded unions of regular pattern languages are polynomial-time predictable with membership query. However, all unbounded unions of regular pattern languages are not polynomial-time predictable with membership queries if neither are the DNF formulas.

Boosting and Hard-Core Set Construction

This paper connects hard-core set construction, a type of hardness amplification from computational complexity, and boosting, a technique from computational learning theory. Using this connection we give fruitful applications of complexity-theoretic techniques to learning theory and vice versa. We show that the hard-core set construction of Impagliazzo (1995), which establishes the existence of distributions under which boolean functions are highly inapproximable, may be viewed as a boosting algorithm. Using alternate boosting methods we give an improved bound for hard-core set construction which matches known lower bounds from boosting and thus is optimal within this class of techniques. We then show how to apply techniques from Impagliazzo (1995) to give a new version of Jackson's celebrated Harmonic Sieve algorithm for learning DNF formulae under the uniform distribution using membership queries. Our new version has a significant asymptotic improvement in running time. Critical to our arguments is a careful analysis of the distributions which are employed in both boosting and hard-core set constructions.