Arbitrary Quantization Research Articles

MBQuant: A novel multi-branch topology method for arbitrary bit-width network quantization

Arbitrary bit-width network quantization has received significant attention due to its high adaptability to various bit-width requirements during runtime. However, in this paper, we investigate existing methods and observe a significant accumulation of quantization errors caused by switching weight and activations bit-widths, leading to limited performance. To address this issue, we propose MBQuant, a novel method that utilizes a multi-branch topology for arbitrary bit-width quantization. MBQuant duplicates the network body into multiple independent branches, where the weights of each branch are quantized to a fixed 2-bit and the activations remain in the input bit-width. For completing the computation of a desired bit-width, MBQuant selects multiple branches, ensuring that the computational costs match those of the desired bit-width, to carry out forward propagation. By fixing the weight bit-width, MBQuant substantially reduces quantization errors caused by switching weight bit-widths. Additionally, we observe that the first branch suffers from quantization errors caused by all bit-widths, leading to performance degradation. Thus, we introduce an amortization branch selection strategy that amortizes the errors. Specifically, the first branch is selected only for certain bit-widths, rather than universally, thereby the errors are distributed among the branches more evenly. Finally, we adopt an in-place distillation strategy that uses the largest bit-width to guide the other bit-widths to further enhance MBQuant’s performance. Extensive experiments demonstrate that MBQuant achieves significant performance gains compared to existing arbitrary bit-width quantization methods. Code is made publicly available at https://github.com/zysxmu/MBQuant.

Joint-Guided Distillation Binary Neural Network via Dynamic Channel-Wise Diversity Enhancement for Object Detection

Through truncating the weights and activations of a deep neural network, conventional binary quantization imposes limitations on the representation capability of the network parameters, which hence deteriorates the detection performance of the network. In this paper, a joint-guided distillation binary neural network via dynamic channel-wise diversity enhancement for object detection (JDBNet) is proposed to mitigate the gap of quantization errors. Our JDBNet includes a dynamic channel-wise diversity scheme and real-valued joint-guided teacher assistance to enhance the representation capability of the binary neural network in the object detection tasks. In the dynamic diversity scheme, the learning channel-wise bias (LCB) layer supports adjusting the magnitude of the parameters in which the sensitivity of the model parameters to the arbitrary quantization method is reduced, thereby improving the diversity expression ability of the feature parameters. In the joint-guided strategy, the single-precision implicit knowledge from the guiding teacher in the multilevel layer is utilized to supervise and penalize the quantitative model, enhancing the fitting performance of parameters in the binary quantized model. Extensive experiments on the PASCAL VOC, MS COCO, and VisDrone-DET datasets demonstrate that our JDBNet outperforms the state-of-the-art binary object detection networks in terms of mean Average Precision.

Multipurpose Deep-Learning Accelerator for Arbitrary Quantization With Reduction of Storage, Logic, and Latency Waste

Multipurpose Deep-Learning Accelerator for Arbitrary Quantization With Reduction of Storage, Logic, and Latency Waste

Adaptive Global Power-of-Two Ternary Quantization Algorithm Based on Unfixed Boundary Thresholds.

In the field of edge computing, quantizing convolutional neural networks (CNNs) using extremely low bit widths can significantly alleviate the associated storage and computational burdens in embedded hardware, thereby improving computational efficiency. However, such quantization also presents a challenge related to substantial decreases in detection accuracy. This paper proposes an innovative method, called Adaptive Global Power-of-Two Ternary Quantization Based on Unfixed Boundary Thresholds (APTQ). APTQ achieves adaptive quantization by quantizing each filter into two binary subfilters represented as power-of-two values, thereby addressing the accuracy degradation caused by a lack of expression ability of low-bit-width weight values and the contradiction between fixed quantization boundaries and the uneven actual weight distribution. It effectively reduces the accuracy loss while at the same time presenting strong hardware-friendly characteristics because of the power-of-two quantization. This paper extends the APTQ algorithm to propose the APQ quantization algorithm, which can adapt to arbitrary quantization bit widths. Furthermore, this paper designs dedicated edge deployment convolutional computation modules for the obtained quantized models. Through quantization comparison experiments with multiple commonly used CNN models utilized on the CIFAR10, CIFAR100, and Mini-ImageNet data sets, it is verified that the APTQ and APQ algorithms possess better accuracy performance than most state-of-the-art quantization algorithms and can achieve results with very low accuracy loss in certain CNNs (e.g., the accuracy loss of the APTQ ternary ResNet-56 model on CIFAR10 is 0.13%). The dedicated convolutional computation modules enable the corresponding quantized models to occupy fewer on-chip hardware resources in edge chips, thereby effectively improving computational efficiency. This adaptive CNN quantization method, combined with the power-of-two quantization results, strikes a balance between the quantization accuracy performance and deployment efficiency in embedded hardware. As such, valuable insights for the industrial edge computing domain can be gained.

Vertical Layering of Quantized Neural Networks for Heterogeneous Inference.

Although considerable progress has been obtained in neural network quantization for efficient inference, existing methods are not scalable to heterogeneous devices as one dedicated model needs to be trained, transmitted, and stored for one specific hardware setting, incurring considerable costs in model training and maintenance. In this paper, we study a new vertical-layered representation of neural network weights for encapsulating all quantized models into a single one. It represents weights as a group of bits (vertical layers) organized from the most significant bit (also called the basic layer) to less significant bits (enhance layers). Hence, a neural network with an arbitrary quantization precision can be obtained by adding corresponding enhance layers to the basic layer. However, we empirically find that models obtained with existing quantization methods suffer severe performance degradation if adapted to vertical-layered weight representation. To this end, we propose a simple once quantization-aware training (QAT) scheme for obtaining high-performance vertical-layered models. Our design incorporates a cascade downsampling mechanism with the multi-objective optimization employed to train the shared source model weights such that they can be updated simultaneously, considering the performance of all networks. After the model is trained, to construct a vertical-layered network, the lowest bit-width quantized weights become the basic layer, and every bit dropped along the downsampling process act as an enhance layer. Our design is extensively evaluated on CIFAR-100 and ImageNet datasets. Experiments show that the proposed vertical-layered representation and developed once QAT scheme are effective in embodying multiple quantized networks into a single one and allow one-time training, and it delivers comparable performance as that of quantized models tailored to any specific bit-width.

Performance Analysis and Optimization of Multicell Massive MIMO With Variable-Resolution ADCs Over Correlated Rayleigh Fading Channels

This article makes performance analysis and optimization for multicell massive multiple-input and multiple-output (MIMO) systems with variable-resolution analog-to-digital converters (ADCs). In such an architecture, every ADC uses arbitrary quantization resolution to save power and hardware cost. Along this direction, we first introduce a quantization-aware channel estimator based on linear minimum mean-squared error (LMMSE) estimate theory over spatially correlated Rayleigh fading channels. By leveraging on the estimated channel state information (CSI), we derive the theoretical expressions of the achievable uplink spectral efficiency (SE) for maximal ratio combining (MRC), quantization-aware multicell minimum mean-squared error (QA-M-MMSE) combining, and quantization-aware single-cell MMSE (QA-S-MMSE) combining, respectively. We consider the impacts of quantization errors and resort to random matrix theory to derive theoretical results. Afterwards, power and bit allocation problems are investigated under the variable-resolution architecture. Finally, simulation results demonstrate that our theoretical analyses are correct and that the proposed quantization-aware estimator and combiners are more beneficial than their quantization-unaware counterparts. Moreover, simulation results also verify the conclusions of the proposed power and bit allocation schemes.

Positivity for quantum cluster algebras from unpunctured orbifolds

We provide a quantum Laurent expansion formula for the quantum cluster algebras from unpunctured orbifolds with arbitrary coefficients and quantization. As an application, positivity for such a class of quantum cluster algebras is provided. Assume for technical reasons that the weights of the orbifold points are 1 / 2 1/2 .

An expansion formula for quantum cluster algebras from unpunctured triangulated surfaces

We study quantum cluster algebras from unpunctured surfaces with arbitrary coefficients and quantization. We first give a new proof of the Laurent expansion formulas for commutative cluster algebras from unpunctured surfaces, we then give the quantum Laurent expansion formulas for the quantum cluster algebras. Particularly, this gives a combinatorial proof of the positivity for such class of quantum cluster algebras.

On the generalization of moyal equation for an arbitrary linear quantization

For an arbitrary linear combination of quantizations, the kernel of the inverse operator is constructed. An equation for the evolution of the Wigner function for an arbitrary linear quantization is derived and it is shown that only for Weyl quantization this equation does not contain a source of quasi-probability. Stationary solutions for the Wigner function of a harmonic oscillator are constructed, depending on the characteristic function of the quantization rule. In the general case of Hermitian linear quantization these solutions are real but not positive. We found the representation of Weyl quantization in the form of the limit of a sequence of linear Hermitian quantizations, such that for each element of this sequence the stationary solution of the Moyal equation is positive.

Discrete Phase Shifters-Based Hybrid Precoding for Full-Duplex mmWave Relaying Systems

This paper considers the hybrid precoding design with quantized phase shifters (PSs) for full-duplex millimeter-wave relaying systems, in which the relay node is equipped with PSs of different resolution levels. The joint optimization of relay hybrid precoding and discrete PSs is a high-dimensional non-convex problem with both higher-order polynomial and modulus constraints. To solve this challenging problem, the optimization function under the imperfect channel state information (CSI) is reformulated into a low-complexity norm-minimization form equation. Then, a relaxed algorithm based on the alternating direction method of multipliers and quantization steps is proposed to obtain the coupling approximate solutions for the above optimized equation. And the optimality of analytical solutions is guaranteed based on the imposed termination criterion and necessary conditions. Furthermore, given the requirement of arbitrary quantization, the trade-off between spectral efficiency (SE) and energy efficiency (EE) is investigated under the imperfect CSI. Numerical results verify that the hybrid precoding with low-resolution PSs can achieve excellent SE close to the full-resolution one but with a considerably lighter burden of energy consumption. Moreover, the transmission power plays a positive role in compensating quantization loss, but the excessive value will bring the reduction of EE.

Generalized Evolution Equation of Wigner Function for an Arbitrary Linear Quantization

The arbitrary linear quantization is considered for definition of Wigner function. The evolution equation of this function is derived with the use of inversion of linear quantization kernel. In general case the Moyal equation depends on the characteristic function of quantization. It is shown that only for Weyl quantization this equation does not consist the source of quasi-probability. The stationary solutions of Moyal equation are constructed for an arbitrary linear quantization of a harmonic oscillator. The case of anharmonic oscillator is presented as a practical example of quantum system model in the so-called post-newtonian approximation. This model exhibits the intersection effect between quantization dependences of Hamiltonian and Wigner function.

Reprogrammable Spatiotemporally Modulated Graphene-Based Functional Metasurfaces

Digital Metasurfaces have recently offered tremendous opportunities in real-time manipulation of terahertz (THz) waves in programmable ways to reach special applications that cannot be achieved using conventional architectures. Unlike the previous demonstrations at this spectrum, which only exploit spatial modulation, in this paper, we propose a spatiotemporally modulated graphene-based digital metasurface (STGDM) to carry out different missions in both space and frequency domains. Applying proper time-variant Fermi level controlling signals turns a 2-bit phase-only meta-atom into a powerful sub-wavelength scatterer with the ability to modulate both phases and amplitudes with arbitrary high quantization levels at center or harmonic frequencies. Several illustrative examples have been presented to demonstrate the excellent capability of our STGDM to dynamically realize different THz scattering behaviors, including beam deflection, vortex beam generation, spatial power dividing, harmonic focusing, and airy beam generation with a single digital interface. The employed space-time coding strategy reveals an additional knob for lifting some fundamental limitations of the previous graphene-based metasurfaces to achieve functionalities requiring both phase and amplitude modulations. Meanwhile, by facilitating the structural design to obtain high quantization levels, the proposed STGDM may be a reliable component in highly-efficient THz imaging and information systems.

Measuring the polarization of electromagnetic fields using Rabi-rate measurements with spatial resolution: Experiment and theory

When internal states of atoms are manipulated using coherent optical or radio-frequency (RF) radiation, it is essential to know the polarization of the radiation with respect to the quantization axis of the atom. We first present a measurement of the two-dimensional spatial distribution of the electric-field amplitude of a linearly-polarized pulsed RF electric field at $\sim 25.6\,$GHz and its angle with respect to a static electric field. The measurements exploit coherent population transfer between the $35$s and $35$p Rydberg states of helium atoms in a pulsed supersonic beam. Based on this experimental result, we develop a general framework in the form of a set of equations relating the five independent polarization parameters of a coherently oscillating field in a fixed laboratory frame to Rabi rates of transitions between a ground and three excited states of an atom with arbitrary quantization axis. We then explain how these equations can be used to fully characterize the polarization in a minimum of five Rabi rate measurements by rotation of an external bias-field, or, knowing the polarization of the driving field, to determine the orientation of the static field using two measurements. The presented technique is not limited to Rydberg atoms and RF fields but can also be applied to characterize optical fields. The technique has potential for sensing the spatiotemporal properties of electromagnetic fields, e.g., in metrology devices or in hybrid experiments involving atoms close to surfaces.

Quantized stabilization of networked control systems with actuator saturation

Summary In this paper, we investigate the problems of stabilization of networked control systems with quantization and actuator saturation via delta operator approach. The definition of the domain of attraction for delta operator systems is introduced to analyze the stochastic stability of the closed-loop networked control systems. The quantizer is a uniform one with arbitrary quantization regions, and the packet dropout process is modeled as a Bernoulli process. On the basis of the zoom strategy and Lyapunov theory in delta domain, sufficient conditions are given for the closed-loop delta operator systems to be mean square stable, and the feedback controllers are designed to ensure the stability of networked control systems. A single link direct joint driven manipulator model is presented to show the effectiveness of the main results. Copyright © 2016 John Wiley & Sons, Ltd.

Quantized global parametrization

Global surface parametrization often requires the use of cuts or charts due to non-trivial topology. In recent years a focus has been on so-called seamless parametrizations, where the transition functions across the cuts are rigid transformations with a rotation about some multiple of 90°. Of particular interest, e.g. for quadrilateral meshing, paneling, or texturing, are those instances where in addition the translational part of these transitions is integral (or more generally: quantized). We show that finding not even the optimal, but just an arbitrary valid quantization (one that does not imply parametric degeneracies), is a complex combinatorial problem. We present a novel method that allows us to solve it, i.e. to find valid as well as good quality quantizations. It is based on an original approach to quickly construct solutions to linear Diophantine equation systems, exploiting the specific geometric nature of the parametrization problem. We thereby largely outperform the state-of-the-art, sometimes by several orders of magnitude.

Graded quiver varieties, quantum cluster algebras and dual canonical basis

Inspired by a previous work of Nakajima, we consider perverse sheaves over acyclic graded quiver varieties and study the Fourier–Sato–Deligne transform from a representation theoretic point of view. We obtain deformed monoidal categorifications of acyclic quantum cluster algebras with specific coefficients. In particular, the (quantum) positivity conjecture is verified whenever there is an acyclic seed in the (quantum) cluster algebra.In the second part of the paper, we introduce new quantizations and show that all quantum cluster monomials in our setting belong to the dual canonical basis of the corresponding quantum unipotent subgroup. This result generalizes previous work by Lampe and by Hernandez–Leclerc from the Kronecker and Dynkin quiver case to the acyclic case.The Fourier transform part of this paper provides crucial input for the second author's paper where he constructs bases of acyclic quantum cluster algebras with arbitrary coefficients and quantization.

Stabilization of fuzzy systems with quantization and packet dropout

SUMMARYThe discrete‐time and continuous‐time fuzzy systems with quantization and packet dropout are considered in this paper. The quantizer used here is a uniform one with arbitrary quantization regions, and the packet dropout process is modeled as a time‐homogenous Markov process. Because of the properties of packet dropout process, the fuzzy systems considered here are seen as Markov jump fuzzy systems, and the theories of Markov jump system are used to discuss the mean square stability of the closed‐loop systems. On the basis of the zoom strategy and Lyapunov theory, for the given failure rate and recovery rate, sufficient conditions are given for the closed‐loop fuzzy systems to be mean square stable, and the feedback controllers are designed to ensure the stability of fuzzy systems. A single link direct joint driven manipulator model is presented to show the effectiveness of the main results. Copyright © 2013 John Wiley & Sons, Ltd.

Quantized Consensus of Multi‐Agent Systems Via Broadcast Gossip Algorithms

AbstractDistributed consensus problems of multi‐agent systems on directed networks are studied in this paper. For the communication of agents, it is assumed that only one agent can be selected with a prescribed probability, and it broadcasts its own state to neighbors via quantized communication (for any arbitrary quantization) at each time step. For this kind of communication, the fundamental questions are how to design distributed algorithms and what kinds of network topology together lead to the quantized consensus. A class of broadcast gossip algorithms is proposed, and a necessary and sufficient graphical condition is given to ensure the quantized consensus. In particular, the obtained graphical condition does not require a symmetric network topology, which is weaker than those in some other literature. Several numerical simulations are given to show the effectiveness of the proposed algorithms.

Rotational symmetry of classical orbits, arbitrary quantization of angular momentum and the role of the gauge field in two-dimensional space

We study quantum—classical correspondence in terms of the coherent wave functions of a charged particle in two-dimensional central-scalar potentials as well as the gauge field of a magnetic flux in the sense that the probability clouds of wave functions are well localized on classical orbits. For both closed and open classical orbits, the non-integer angular-momentum quantization with the level space of angular momentum being greater or less than ħ is determined uniquely by the same rotational symmetry of classical orbits and probability clouds of coherent wave functions, which is not necessarily 2π-periodic. The gauge potential of a magnetic flux impenetrable to the particle cannot change the quantization rule but is able to shift the spectrum of canonical angular momentum by a flux-dependent value, which results in a common topological phase for all wave functions in the given model. The well-known quantum mechanical anyon model becomes a special case of the arbitrary quantization, where the classical orbits are 2π-periodic.

Robust watermarking procedure based on JPEG discrete cosine transform image compression

A new procedure for watermarking in the 88 block- based discrete cosine transform (DCT) domain (used for JPEG compression) is proposed. The influence of JPEG quantization on watermarked coefficients and on watermark is considered. The cri- terion for coefficients selection is derived, providing robustness for an arbitrary quantization degree. The modified form of coefficients probability density function (pdf) leads to the class of modified opti- mal detectors. Theoretical results are illustrated on various ex- amples. Efficiency of the proposed procedure is shown in the pres- ence of different quantization degrees, and other common attacks. © 2008 SPIE and IS&T. DOI: 10.1117/1.2981832