Threshold Circuits Research Articles

Trade-Offs Between Energy and Depth of Neural Networks.

We present an investigation on threshold circuits and other discretized neural networks in terms of the following four computational resources-size (the number of gates), depth (the number of layers), weight (weight resolution), and energy-where the energy is a complexity measure inspired by sparse coding and is defined as the maximum number of gates outputting nonzero values, taken over all the input assignments. As our main result, we prove that if a threshold circuit C of size s, depth d, energy e, and weight w computes a Boolean function f (i.e., a classification task) of n variables, it holds that log( rk (f))≤ed(logs+logw+logn) regardless of the algorithm employed by C to compute f, where rk (f) is a parameter solely determined by a scale of f and defined as the maximum rank of a communication matrix with regard to f taken over all the possible partitions of the n input variables. For example, given a Boolean function CD n(ξ) =⋁i=1n/2ξi∧ξn/2+i, we can prove that n/2≤ed( log s+logw+logn) holds for any circuit C computing CD n. While its left-hand side is linear in n, its right-hand side is bounded by the product of the logarithmic factors of s,w,n and the linear factors of d,e. If we view the logarithmic terms as having a negligible impact on the bound, our result implies a trade-off between depth and energy: n/2 needs to be smaller than the product of e and d. For other neural network models, such as discretized ReLU circuits and discretized sigmoid circuits, we also prove that a similar trade-off holds. Thus, our results indicate that increasing depth linearly enhances the capability of neural networks to acquire sparse representations when there are hardware constraints on the number of neurons and weight resolution.

Aptamer-Based DNA Allosteric Switch for Regulation of Protein Activity.

Allostery is a fundamental way to regulate the function of biomolecules playing crucial roles in cell metabolism and proliferation and is deemed the second secret of life. Given the limited understanding of the structure of natural allosteric molecules, the development of artificial allosteric molecules brings a huge opportunity to transform the allosteric mechanism into practical applications. In this study, the concept of bionics is introduced into the design of artificial allosteric molecules and an allosteric DNA switch with an activity site and an allosteric site based on two aptamers for selective inhibition of thrombin activity. Compared with the single aptamer, the allosteric switch possesses a significantly enhanced inhibition ability, which can be precisely regulated by converting the switch states. Moreover, the dynamic allosteric switch is further subjected to the control of the DNA threshold circuit for realizing automatic concentration determination and activity inhibition of thrombin. These compelling results confirm that this allosteric switch equipped with self-sensing and information-processing modules puts a new slant on the research of allosteric mechanisms and further application of allosteric tactics in chemical and biomedical fields.

Threshold Circuit Based on Cyclic Codes

Threshold Circuit Based on Cyclic Codes

The Parallelism Tradeoff: Limitations of Log-Precision Transformers

AbstractDespite their omnipresence in modern NLP, characterizing the computational power of transformer neural nets remains an interesting open question. We prove that transformers whose arithmetic precision is logarithmic in the number of input tokens (and whose feedforward nets are computable using space linear in their input) can be simulated by constant-depth logspace-uniform threshold circuits. This provides insight on the power of transformers using known results in complexity theory. For example, if L≠P (i.e., not all poly-time problems can be solved using logarithmic space), then transformers cannot even accurately solve linear equalities or check membership in an arbitrary context-free grammar with empty productions. Our result intuitively emerges from the transformer architecture’s high parallelizability. We thus speculatively introduce the idea of a fundamental parallelism tradeoff: any model architecture as parallelizable as the transformer will obey limitations similar to it. Since parallelism is key to training models at massive scale, this suggests a potential inherent weakness of the scaling paradigm.

Linear threshold functions in decision lists, decision trees, and depth-2 circuits

We show that polynomial-size constant-rank linear decision trees (LDTs) can be converted to polynomial-size depth-2 threshold circuits LTF∘LTF. An intermediate construct is polynomial-size decision lists that query a conjunction of a constant number of linear threshold functions (LTFs); we show that these are equivalent to polynomial-size exact linear decision lists (ELDLs) i.e. decision lists querying exact threshold functions (ELTFs).

Parameterized Complexity Classes Defined by Threshold Circuits and Their Connection with Sorting Networks

The main complexity classes of the Parameterized Intractability Theory are based on weighted Boolean circuit satisfiability problems and organized into a hierarchy so-called W-hierarchy. The W-hierarchy enables fine-grained complexity analyses of parameterized problems that are unlikely to belong to the FPT class. In this paper, we introduce the Th-hierarchy, a natural generalization of the W-hierarchy defined by unweighted threshold circuit satisfiability problems. Investigating the relationship between Th-hierarchy and W-hierarchy, we discuss the complexity of transforming Threshold circuits into Boolean circuits, and observe that sorting networks are powerful tools to handle such transformations. First, we show that these hierarchies collapse at the last level (W[P][Formula: see text][Formula: see text][Formula: see text]Th[P]). After that, we present a time complexity analysis of an AKS sorting network construction, which supports some of our results. Finally, we prove that Th[[Formula: see text]] [Formula: see text] W[SAT] for every [Formula: see text]. As a by-product, our studies suggest that it is relevant to consider a new class based on logarithmic depth circuits in the W-hierarchy.

Design and Implementation of CNFET SRAM Cells by Using Multi-Threshold Technique

This paper presents a CNFET (Carbon Nano-tube FET) based MT (Multi-Threshold)-SRAM (Static Random Access Memory) design based on the leakage reduction mechanism. A multi-threshold logic is employed for reducing the leakage current during read/write operations. Here, the multi-threshold technique is used to insert the high threshold sleep control to the low threshold circuit. The insertion is performed in a serial manner. The high threshold transistors are very useful for deriving the low sub-threshold current. Meanwhile, the low threshold transistors are promising for improving the circuit performance. The high-low threshold transistor pairs are used to change the channel length by modifying the oxide thickness of the transistors. The overall implementation of the Multi-threshold-based SRAM cells are implemented with the help of CNFET in-order to avoid the short channel effect, mobility degradation which is occurred while considering the channel length below 32 nm in CMOS (Complementary Metal Oxide Semiconductor) devices. The paper clearly represents the performance improvement of the proposed SRAM cells with above-mentioned technologies.

A Robust Version of Heged\H{u}s's Lemma, with Applications

Heged\H{u}s's lemma is the following combinatorial statement regarding polynomials over finite fields. Over a field $\mathbb{F}$ of characteristic $p > 0$ and for $q$ a power of $p$, the lemma says that any multilinear polynomial $P\in \mathbb{F}[x_1,\ldots,x_n]$ of degree less than $q$ that vanishes at all points in $\{0,1\}^n$ of some fixed Hamming weight $k\in [q,n-q]$ must also vanish at all points in $\{0,1\}^n$ of weight $k + q$. This lemma was used by Heged\H{u}s (2009) to give a solution to \emph{Galvin's problem}, an extremal problem about set systems; by Alon, Kumar and Volk (2018) to improve the best-known multilinear circuit lower bounds; and by Hrube\v{s}, Ramamoorthy, Rao and Yehudayoff (2019) to prove optimal lower bounds against depth-$2$ threshold circuits for computing some symmetric functions. In this paper, we formulate a robust version of Heged\H{u}s's lemma. Informally, this version says that if a polynomial of degree $o(q)$ vanishes at most points of weight $k$, then it vanishes at many points of weight $k+q$. We prove this lemma and give three different applications.

Computing the best-case energy complexity of satisfying assignments in monotone circuits

Measures of circuit complexity are usually analyzed to ensure the computation of Boolean functions with economy and efficiency. One of these measures is the energy complexity, which is related to the number of gates that output true in a circuit for an assignment. The idea behind energy complexity comes from the counting of ‘firing’ neurons in a natural neural network. The initial model is based on threshold circuits, but recent works also have analyzed the energy complexity of traditional Boolean circuits. In this work, we discuss the time complexity needed to compute the best-case energy complexity among satisfying assignments of a monotone Boolean circuit, and we call such a problem as MinECM+. In the MinECM+ problem, we are given a monotone Boolean circuit C, a positive integer k and asked to determine whether there is a satisfying assignment X for C such that EC(C,X)≤k, where EC(C,X) is the number of gates that output true in C according to the assignment X. We prove that MinECM+ is NP-complete even when the input monotone circuit is planar. Besides, we show that the problem is W[1]-hard but in XP when parameterized by the size of the solution. In contrast, we show that when the size of the solution and the genus of the input circuit are aggregated parameters, the MinECM+ problem becomes fixed-parameter tractable.

Saturated Transformers are Constant-Depth Threshold Circuits

Abstract Transformers have become a standard neural network architecture for many NLP problems, motivating theoretical analysis of their power in terms of formal languages. Recent work has shown that transformers with hard attention are quite limited in power (Hahn, 2020), as they can be simulated by constant-depth AND/OR circuits (Hao et al., 2022). However, hard attention is a strong assumption, which may complicate the relevance of these results in practice. In this work, we analyze the circuit complexity of transformers with saturated attention: a generalization of hard attention that more closely captures the attention patterns learnable in practical transformers. We first show that saturated transformers transcend the known limitations of hard-attention transformers. We then prove saturated transformers with floating-point values can be simulated by constant-depth threshold circuits, giving the class TC0 as an upper bound on the formal languages they recognize.

Quantum Tunneling Based Ultra-Compact and Energy Efficient Spiking Neuron Enables Hardware SNN

Low-power and low-area neurons are essential for hardware implementation of large-scale SNNs. Various novel-physics-based leaky-integrate-and-fire (LIF) neuron architectures have been proposed with low power and area, but are not compatible with CMOS technology to enable brain scale implementation of SNN. In this paper, for the first time, we demonstrate hardware implementation of recurrent SNN using proposed low-power, low-area, and low-leakage band-to-band-tunneling (BTBT) based neurons. A low-power thresholding circuit is proposed. We further propose a predistortion technique to linearize a nonlinear neuron without any area and power overhead. We establish the equivalence of the proposed neuron with the ideal LIF neuron to demonstrate its versatility. The tunneling regime enables a high input impedance in the BTBT neurons (few <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{G}\Omega$ </tex-math></inline-formula> ) to enable a voltage input without loading the synaptic array. To verify the effect of the proposed neuron, a 36-neuron recurrent SNN is fabricated in GF-45nm PDSOI technology. We achieved 5000x lower energy-per-spike at a similar area and 10x lower standby power at a similar area and energy-per-spike. Such overall performance improvement enables brain scale computing.

Assessment of the impact of interference signals on the decision circuit of the receiving-analyzing route of digital radio engineering systems

Currently, due to the increasing complexity of navigation support for aircraft (A/C), with the growth of requirements for them, it is increasingly necessary to develop systems for integrated processing of navigation information. The facilities for radio engineering flights support and aeronautical telecommunications are the main source of collecting processing and presenting information about the air situation to the traffic control staff. They are also used to solve the number of navigation problems. At the same time, the level of flight safety, along with the capacity of air traffic control zones depending on the decision of the ATC unit, is largely specified by the reliability of the various input information provided. At another point, most sophisticated digital tools for radio engineering flights support and aeronautical telecommunications at the output of the receiving-analyzing route have a decision or threshold circuit, the actuation of which, as a result of exposure to a mixture of noise and interference signals, reduces the reliability of information due to bursts of signal distribution, provided that impulse interference affecting the equipment and mush of receivers are not correlated. The work provides an estimate of the average amount of signal bursts at the output of the decision circuit taking into consideration the analysis of the probabilistic characteristics of the signals under consideration. For noises subject to the Rayleigh distribution and narrow-band impulse interference with a zero-mean value, the mathematical expectations of the average response time and the average number of positive bursts of the estimated process are found. The case of interfacing two airway surveillance radars “Skala-M” and “ATCR-22”, which have virtually identical operating characteristics, is analyzed. The dependence of the burst duration is calculated for the situation when the “Skala-M” radar is a source of unintended EMI for the “ATCR-22” radar.

A Review on Principles, Theories and Materials for Self Sensing Concrete for Structural Applications.

Self-sensing concrete is a smart material known for its cost-effectiveness in structural health-monitoring areas, which converts the external stimuli into a stress/strain sensing parameter. Self-sensing material has excellent mechanical and electrical properties that allow it to act as a multifunctional agent satisfying both the strength and structural health-monitoring parameters. The main objective of this review is to understand the theories and principles behind the self-sensing practices. Many review papers have focused on the different types of materials and practices that rely on self-sensing technology, and only a few articles have discussed the theories involved. Understanding the mechanism and the theories behind the conduction mechanism is necessary. This review paper provides an overview of self-sensing concrete, including the principles such as piezoresistivity and piezopermittivity; the tunnelling effect, percolation threshold, and electrical circuit theories; the materials used and methods adopted; and the sensing parameters. The paper concludes with an outline of the application of self-sensing concrete and future recommendations, thus providing a better understanding of implementing the self-sensing technique in construction.

Line average wind speed measurement system of roadway cross-section based on ultrasonic transit time Difference

Aiming at the problems of low measurement accuracy, poor adaptability, and lack of real-time online measurement methods for roadway section average wind speed, combined with the characteristics of roadway section, based on the characteristics of time difference caused by ultrasonic propagation in forward and reverse flow medium, high-energy ultrasonic transmission and long-distance transit are realized by adopting automatic tracking correction technology of transmission frequency and high-pressure pulse drive technology. program-controlled filter amplification technology and peak hold technology are used to achieve high quality processing of received signals. Accurate measurement of ultrasonic transit time is realized by adopting AD high-speed sampling time positioning technology, floating threshold time difference measurement technology and circuit time delay self-determination measurement technology, and finally realizes the measurement of the average wind speed of the roadway cross-section line. The measurement system is tested on the gas flow standard device, and the measurement accuracy meets and exceeds the requirements of relevant standards. Compared with the current technical means for measuring cross-section wind speed in roadways, the system has the advantages of one-time measurement of average line wind speed, high accuracy, strong adaptability and real-time on-line monitoring.

A #SAT Algorithm for Small Constant-Depth Circuits with PTF gates

We show that there is a better-than-brute-force algorithm that, when given a small constant-depth Boolean circuit C made up of gates that compute constant-degree Polynomial Threshold functions or PTFs (i.e., Boolean functions that compute signs of constant-degree polynomials), counts the number of satisfying assignments to C in significantly better than brute-force time. Formally, for any constants d, k, there is an \(\varepsilon > 0\) such that the zero-error randomized algorithm counts the number of satisfying assignments to a given depth-d circuit C made up of k-PTF gates such that C has at most \(n^{1+\varepsilon }\) many wires. The algorithm runs in time \(2^{n-n^{\Omega (\varepsilon )}}.\) Before our result, no algorithm for beating brute-force search was known for counting the number of satisfying assignments even for a single degree-k PTF (which is a depth-1 circuit with linearly many wires).We give two different algorithms for the case of a single PTF. The first uses a learning algorithm for learning degree-1 PTFs (or Linear Threshold Functions) using comparison queries due to Kane, Lovett and Moran (STOC 2018), and the second uses a proof of Hofmeister (COCOON 1996) for converting a degree-1 PTF to a depth-two threshold circuit with small weights. We show that both these ideas fit nicely into a memoization approach that yields the #SAT algorithms.

A High-Speed Bidirectional Register with Parallel Loading using single electron Threshold Logic Technology

In this work we have concentrated our attention to a High Speed 4-bit Bidirectional Register with Parallel Loading counting on the principle of threshold logic gates (TLG). After determining the number of logic gates and other circuits needed to complete the desired circuit for our work, we implement some gates and circuits made up of tunnel junctions and capacitances. Some multi-inputs (greater than two) are designed or implemented with the assistance of modified version of the generic multi-input TLG. The types of gates suitable for the implementing the bidirectional Register are 3-input AND, 3-input NAND and 4-input OR gates, in addition an inverter and a more complex circuits like 4:1 Multiplexer are the part and parcel of the desired device. With the help of a 3-input AND gate and a 4-input OR gate, a 4:1 Multiplexer is built. By using the 3-input NAND gate a memory element – D Flip-flop is constructed. At last 4 number of 4:1 Multiplexers and another four number of D Flip-flops are combined in a parallel pattern to implement a 4-bit Bidirectional Register with Parallel Loading. Each component is made after analyzing their corresponding threshold linear equations. After constructing the threshold circuits, again they are formed by using the parameters as capacitors, tunnel junctions with their internal resistances. All the circuit, which are constructed, are verified by simulation with the help of SIMON and the result obtained are investigated and found that they are matched with the theoretical results. For comparing the fastness of our circuit with the CMOS-based or single electron transistor (SET) based circuit, the processing delays of all gates/ circuits are determined. How much power they consume are measured as well. Comparing the delays of CMOS-based and SET based circuit with the TLG based circuit we have decided that our 4-bit Bidirectional Register with Parallel Loading is speedier.

Design of Photovoltaic Panel-Light-Emitting Diode Spotlight and Its Performance Analysis in Tunnel Buildings

According to the adaptive characteristics of the entrance and exit sections of tunnel buildings with the intensity of external sunlight, the cost of traditional photovoltaic panel (PV)-light-emitting diode (LED) lighting system increases due to battery. Combined with the structure of sunshine conveyor, a new PV-LED spotlight is proposed. The lighting device uses PV module as the power supply to directly drive the LED light source, which can make the brightness of the tunnel synchronized with the sunlight intensity, avoid the problem that it cannot work stably due to its weather changes, and further save the cost without battery. The main control chip adopts STC12C5A60S2 and supports multiple 10-bits precision A/D conversion channels. It can process the output signal of the light intensity detection circuit and set a threshold circuit to judge the intensity of sunlight, further control the working state of LED lighting device. The LED display module adopts 12864 liquid crystal display (LCD) with built-in T6963C driver to display five kinds of data, including the output voltage of light intensity detection circuit and the corresponding voltage value of four threshold circuits. The circuit resistance can be adjusted to change the light output voltage threshold required by the tunnel. In the test, the PV-LED lighting device is erected in the tunnel building. Compared with the traditional PV-LED lighting device and the new PV-LED spotlight designed, the new PV-LED lighting device in the morning and evening can further improve the efficiency of solar power generation. Meanwhile, the PV plate plane of the designed lighting device is perpendicular to the plane where the sunlight shines. The spectral irradiance and full band spectral irradiance of the lighting device in six bands was calculated and found that the percentage of spectral irradiance in different bands is close to the percentage requirement under AM1.5.

An Optoelectronic Targeting System for Measuring the Distribution of Projectile Motion Based on the Subdivision of a Light Screen

This paper proposes a cost-effective, compact, noncontacting optoelectronic targeting system for measuring the distribution of projectile motion. The major elements of this system include a light emitting diode (LED) array, photodiode detecting array, double-layered aperture arrays, adaptive threshold circuit, and date acquisition. Through cooperating with double-layered aperture arrays, the system effectively reduces the radiation width of the light source to the photodiode detecting surface, and filters out the influence of incident light from the adjacent apertures on both sides above each photodiode to the corresponding photodiode detecting surface. It realizes that the response of the photodiode array corresponds to the coordinates of the light screen one by one. Through the sensitivity analysis of the light screen of the system, the system detecting threshold when the projectile passes through the light screen is calculated, and the corresponding adaptive threshold circuit is designed to prevent misjudgment when the system works. The measuring error of the system can reach ±2 mm by experimental verification. Compared with other projectile’s distribution measuring systems, the proposed system has the advantages of having high precision, convenient debugging, is nondestructive, and is a noncontact system.

Depth-2 Threshold Circuits

Circuits with linear threshold functions as primitives are a natural model for computation in the brain. Small threshold circuits of depth two cannot compute most functions, but how do we prove such a statement? And how do we lay our hands on explicit functions that they cannot compute? This article gives an overview of the landscape.

On the Representational Power of Restricted Boltzmann Machines for Symmetric Functions and Boolean Functions.

Restricted Boltzmann machines (RBMs) are used to build deep-belief networks that are widely thought to be one of the first effective deep learning neural networks. This paper studies the ability of RBMs to represent distributions over {0,1}n via softplus/hardplus RBM networks. It is shown that any distribution whose density depends on the number of 1's in their input can be approximated with arbitrarily high accuracy by an RBM of size 2n+1 , which improves the result of a previous study by reducing the size from n2 to 2n+1 . A theorem for representing partially symmetric Boolean functions by softplus RBM networks is established. Accordingly, the representational power of RBMs for distributions whose mass represents the Boolean functions is investigated in comparison with that of threshold circuits and polynomial threshold functions. It is shown that a distribution over {0,1}n whose mass represents a Boolean function can be computed with a given margin δ by an RBM of size and parameters bounded by polynomials in n , if and only if it can be computed by a depth-2 threshold circuit with size and parameters bounded by polynomials in n .