Families Of Boolean Circuits Research Articles

Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity

Abstract This paper analyzes three formal models of Transformer encoders that differ in the form of their self-attention mechanism: unique hard attention (UHAT); generalized unique hard attention (GUHAT), which generalizes UHAT; and averaging hard attention (AHAT). We show that UHAT and GUHAT Transformers, viewed as string acceptors, can only recognize formal languages in the complexity class AC0, the class of languages recognizable by families of Boolean circuits of constant depth and polynomial size. This upper bound subsumes Hahn’s (2020) results that GUHAT cannot recognize the DYCK languages or the PARITY language, since those languages are outside AC0 (Furst et al., 1984). In contrast, the non-AC0 languages MAJORITY and DYCK-1 are recognizable by AHAT networks, implying that AHAT can recognize languages that UHAT and GUHAT cannot.

On Symmetric Circuits and Fixed-Point Logics

We study properties of relational structures, such as graphs, that are decided by families of Boolean circuits. Circuits that decide such properties are necessarily invariant to permutations of the elements of the input structures. We focus on families of circuits that are symmetric, i.e., circuits whose invariance is witnessed by automorphisms of the circuit induced by the permutation of the input structure. We show that the expressive power of such families is closely tied to definability in logic. In particular, we show that the queries defined on structures by uniform families of symmetric Boolean circuits with majority gates are exactly those definable in fixed-point logic with counting. This shows that inexpressibility results in the latter logic lead to lower bounds against polynomial-size families of symmetric circuits.

Highly parallel computations modulo a number having only small prime factors

Highly parallel algorithms computing the inverse, discrete roots, or a large power modulo a number that has only small prime factors are presented. The elaborated uniform families of Boolean circuits simultaneously achieve depth O (log n ) and size O ( n 0(1) ) for P -uniformity and depth O (log n log log n ) and size O ( n 0(1) ) for log-space uniformity.

A div(n) depth Boolean circuit for smooth modular inverse

We present two families of Boolean circuits of depth O(log n) and size n O(1) (P uniform), and of depth O(log n log log n) and size n O(1) (log-space uniform) for computing the modular inverse of an n-bit integer, where the modulus has only small prime factors. More precisely, this paper provides an NC 1 reduction of the problem mentioned above to integer division with remainder. In the case of P uniformity the circuits presented are asymptotically best possible.

Very Fast Parallel Polynomial Arithmetic

Parallel algorithms for polynomial arithmetic—multiplication of n polynomials of degree m, polynomial division with remainder, and polynomial interpolation—are presented. These algorithms can be implemented using polynomial time constructible families of Boolean circuits of polynomial size and optimal order depth, or log space constructible families of polynomial size and near optimal depth, for computations over $\mathbb{Z},\mathbb{Q}$, finite fields, and several other domains. Arithmetic circuits of polynomial size and optimal order depth are obtained for polynomial arithmetic over arbitrary fields.