Uniform Quantization Research Articles

A Gray Scale Image Compression Using Hierarchical DWT Decomposition and Mixing Quantization Techniques

This paper describes a hybrid grayscale compression system based on the discrete wavelet transform (DWT) and a polynomial coding technique for mixing quantization schemes to increase the compression ratio while maintaining quality. The proposed compression system consists of three main steps: first, decomposing an image using a three-level DWT; second, applying the 2-D linear polynomial coding technique to the approximation sub-band; and third, dividing the detailed sub-bands of each level into the Most Significant Value (MSV) and Least Significant Value (LSV), where the former is compressed using iterative scalar uniform quantization and the latter by soft quantization thresholding. For testing the performance of the suggested compression system, five standard images of size 256×256 pixels were adopted. The suggested technique showed superior performance in terms of reconstructed (decoded) image quality and compression ratio (gain), where the compression ratio is between 21–27 with a PSNR value between 36–38 dB and the compression ratio of JPEG is between 7–20 with a PSNR value between 33–37 dB.

Distributed adaptive moving horizon estimation for multi-sensor networks subject to quantization effects

This paper investigates the distributed state estimation problem for multi-sensor networks with quantized measurements. Within the Bayesian framework, a distributed adaptive moving horizon estimation algorithm is developed. Unlike the existing methods regarding quantized errors roughly as bounded uncertainties, the posterior distributions of the errors are demanded to be derived. To overcome the difficulty of evaluating the posterior distributions for series of the states and quantized errors jointly, the variational Bayesian methodology is adopted to approximate the true distributions. Based on the fixed-point iteration method, the update rules are analytically derived, with the convergence criterion provided. Furthermore, by incorporating the average consensus algorithm into the prediction process, all sensors can achieve consensus on their estimates in a distributed manner. Finally, a numerical example of target tracking under logarithmic and uniform quantization effects is given to illustrate the validity of the proposed algorithm.

Quantization-Based Adaptive Fuzzy Consensus for Multiagent Systems Under Sensor Deception Attacks: A Novel Compensation Mechanism.

This study mainly investigates the adaptive leader-following consensus tracking control problem for a class of nonlinear multiagent systems (MASs) subjected to unknown control directions, external disturbances, and sensor deception attacks. To start with, an equivalent MAS with known control directions is obtained by introducing a linear state transformation. For the purpose of estimating the unavailable system states caused by malicious attacks, a quantization-based fuzzy state observer is designed, and the fuzzy-logic system (FLS) is utilized to approximate nonlinear functions. Moreover, a dynamic uniform quantizer with scaling function is established to reduce information transmission. With the help of coordinate transformation and available compromised states, a novel compensation mechanism is designed to offset the influence of filter errors while avoiding the problem of "explosion of complexity" in the backstepping design process. In addition, the Nussbaum-type function is considered to eliminate the design obstacle of unknown control gains resulting from the attacks. Under the constructed consensus protocol, it is proved theoretically that the consensus tracking error converges to an adjustable small neighborhood of the origin, and all signals in the closed-loop system are bounded. Finally, the feasibility of the provided secure control scheme is verified through two simulation examples.

Low complexity and finer granularity quantization scheme for perturbation-based fiber nonlinearity compensation in coherent optical communication systems

In the efficient implementation of perturbation theory-based nonlinearity compensation method for reliable fiber-optic coherent communication systems, different quantization schemes have been proposed to reduce the required computational and implementation complexity. For example, the k-means clustering algorithm focused on reducing the number of perturbation coefficients to alleviate the burden of calculating the perturbation terms. In this paper, from the bit-level architectures of multipliers, we propose a multi-segment mixed-word-length quantization scheme that assigns different word lengths for the quantized perturbative coefficients to save computational resources, where Bayesian optimization is employed to derive the optimal hyperparameters involved in this scheme. The proposed scheme was evaluated by numerical simulations of single-channel and multi-channel optical fiber communication systems with dual-polarization 16-QAM modulation. The comprehensive numerical simulation results show that our proposed scheme demonstrates a complexity reduction of more than 65% without performance degradation compared with the traditional uniform quantization method with fixed-word-length, and was robust to the fiber length and baud-rate. We also carried out an experiment of single-carrier transmission over 18 × 100 km standard single-mode fiber with 20 Gbaud single-polarization 16-QAM signal. Similar to the numerical results, the hardware complexity can be reduced by 70.7% with negligible performance deterioration over the uniform quantization scheme. The proposed scheme shows great potential in real-world applications.

Adaptive non-iterative histogram-based hologram quantization

Quantization is widely used for compression, storage, transmission and display of computer-generated and digital holograms. It reduces number of hologram gradations to obtain better ratio of the quality of reconstructed 2D and 3D objects to the size of holographic images or videos. This paper proposes 2 methods of non-iterative quantization of holograms: adaptive non-uniform quantization based on histogram processing with peak shift (ANHS method) and additional application of logarithmic companding (ANHSL method). They were compared with 4 methods of iterative and non-iterative quantization on holograms with sizes up to 2048 × 2048 pixels. The proposed methods showed the best reconstruction quality that is higher up to 59 % in comparison with non-iterative uniform quantization. Especially the proposed methods give higher quality at little number of hologram gradations (from 2 to 16 gradations). In comparison with the most qualitative iterative methods (Lloyd-Max and k-means) the quality is increased up to 9 %, and processing time is reduced in 10–1000 times.

High Performance CNN Accelerators based on Hardware and Algorithm Co-Optimization

The efficacy of convolutional neural networks (CNNs) has led to their extensive application in picture recognition and categorization. Nevertheless, embedded systems face challenges when it comes to storing the massive volumes of weight data required by CNNs in their on-chip memory. Pruning a convolutional neural network (CNN) model can reduce its size with no impact on accuracy; however, when used with a parallel architecture, the resulting model will be slower to execute. A CNN compression method that is hardware-centric is introduced in this paper. A model of a deep neural network (DNN) will include "pruning layers (P-layers)" and "no-pruning layers (NP-layers)" attached to it. For effective and parallel processing, an NP-layer employs a regular distribution for its weights. Pruning causes a P-layer to have an irregular form, yet the high compression ratio it yields makes the shape undesirable. Using uniform and gradual quantization methods, we can achieve a processing efficiency/compression ratio trade-off with a small loss of precision. Integrating multiple parallel finite impulse response (FIR) filters into an NP-layer distributed convolutional architecture, we further improve the regular model. The P-layer irregular sparse model is suggested to use an ADF-based processing element. The suggested compression strategy and hardware architecture can be utilised by a hardware/algorithm co-optimization (HACO) method to run an NP→P hybrid compressed CNN model on FPGAs. The VGG-16 network achieves a compression ratio of 27.5× with a top-5 accuracy loss of 0.44% by using a hardware accelerator on a single FPGA chip without off-chip memory. A result of 83.0 FPS, or 1.8 times quicker, was achieved when the compressed VGG-16 model was implemented on a Xilinx VCU118 evaluation board for image applications. Index Terms - CNN, Accelerators, Co-Optimization.

Distributed adaptive finite-time output feedback containment control for nonstrict-feedback stochastic multi-agent systems via command filters

In this paper, distributed adaptive finite-time output feedback containment control based on command filters is proposed for nonstrict-feedback stochastic multi-agent systems (SMASs) with unmodeled dynamics, input quantization and prescribed performance. The unknown states are estimated using a high-gain observer, the unmodeled dynamics is handled by introducing a measurable dynamic signal, and the input signal is processed using an integrated quantizer that combines the advantages of uniform quantizer and hysteretic quantizer. The hyperbolic tangent function and time-varying function are applied to construct the performance function, and the constrained containment error is transformed into an unconstrained system through nonlinear mapping (NM). Unknown smooth functions are handled with the superb approximation capability of radial basis function neural networks (RBFNNs). Based on stochastic finite-time theory and dynamic surface control (DSC) method, nonlinear command filters are introduced to reduce the use of black-box functions and simplify the controller design. Finally, the theoretical analysis and two sets of experimental results demonstrate that all signals in the controlled system are semi-global finite-time stability in probability (SGFSP) and the designed controller works well.

A Convolutional Neural Network-Based Quantization Method for Block Compressed Sensing of Images.

Block compressed sensing (BCS) is a promising method for resource-constrained image/video coding applications. However, the quantization of BCS measurements has posed a challenge, leading to significant quantization errors and encoding redundancy. In this paper, we propose a quantization method for BCS measurements using convolutional neural networks (CNN). The quantization process maps measurements to quantized data that follow a uniform distribution based on the measurements' distribution, which aims to maximize the amount of information carried by the quantized data. The dequantization process restores the quantized data to data that conform to the measurements' distribution. The restored data are then modified by the correlation information of the measurements drawn from the quantized data, with the goal of minimizing the quantization errors. The proposed method uses CNNs to construct quantization and dequantization processes, and the networks are trained jointly. The distribution parameters of each block are used as side information, which is quantized with 1 bit by the same method. Extensive experiments on four public datasets showed that, compared with uniform quantization and entropy coding, the proposed method can improve the PSNR by an average of 0.48 dB without using entropy coding when the compression bit rate is 0.1 bpp.

Sequential Fusion Estimation for Multirate Complex Networks With Uniform Quantization: A Zonotopic Set-Membership Approach.

In this article, the sequential fusion estimation problem is investigated for multirate complex networks (MRCNs) with uniformly quantized measurements. The process and measurement noises, which are unknown-yet-bounded (UYB), are restrained into a family of zonotopes, and the multiple sensors are allowed to have different sampling periods. To facilitate digital transmissions, the sensor measurements are uniformly quantized before being sent to the remote estimator. The purpose of this article is to design a sequential set-membership estimator such that, in the simultaneous presence of UYB noises, multirate samplings, and uniform quantization effects, the estimation error (after each measurement update) is confined to a zonotope with minimum F -radius at each time instant. By introducing certain virtual measurements, the MRCNs are first transformed into single-rate ones exhibiting a switching phenomenon. Then, by utilizing the properties of zonotopes, the desired zonotopes are derived, which contain the estimation error dynamics after each measurement update. Subsequently, the gain matrices of the sequential estimator are derived by minimizing the F -radii of these zonotopes, and the uniform boundedness is analyzed for the F -radius of the zonotope containing the estimation error after all measurement updates. Furthermore, sufficient conditions are derived to ensure the existence of the desired uniform upper/lower bounds. Finally, an illustrated example is proposed to show the effectiveness of the proposed sequential fusion estimation method.

Distributed Event-Triggered Quantized Fault-Tolerant Control of Linear Multiagent Systems With External Disturbances and Parameter Uncertainties.

In this article, the issue of fault-tolerant leader-following consensus under a distributed dynamic event-triggered mechanism is addressed for linear multiagent systems (MASs) in the presence of unknown parameter uncertainties, external disturbances, and actuator faults, including loss of effectiveness and bias, in which the mechanism is with quantized state measurements. Due to the fact that information is transmitted via a bandwidth-limited communication network, a quantized control scheme with a uniform quantizer is introduced for leader-following consensus. In order to decrease the communication load and save the limited communication network resources, a distributed event-triggered mechanism is studied for leader-following consensus problem of linear MASs with quantized state measurements. In the presence of actuator faults, external disturbances, and unknown parameter uncertainties, an adaptive coupling gain for the controller is presented. Based on the Lyapunov function approach, the stability of the closed-loop system and the convergence of consensus errors are proved. Furthermore, the Zeno behavior is excluded for the triggering time sequences. Finally, simulation studies are given to verify the effectiveness of the proposed event-triggered fault-tolerant control scheme.

PTMQ: Post-training Multi-Bit Quantization of Neural Networks

The ability of model quantization with arbitrary bit-width to dynamically meet diverse bit-width requirements during runtime has attracted significant attention. Recent research has focused on optimizing large-scale training methods to achieve robust bit-width adaptation, which is a time-consuming process requiring hundreds of GPU hours. Furthermore, converting bit-widths requires recalculating statistical parameters of the norm layers, thereby impeding real-time switching of the bit-width. To overcome these challenges, we propose an efficient Post-Training Multi-bit Quantization (PTMQ) scheme that requires only a small amount of calibration data to perform block-wise reconstruction of multi-bit quantization errors. It eliminates the influence of statistical parameters by fusing norm layers, and supports real-time switching bit-widths in uniform quantization and mixed-precision quantization. To improve quantization accuracy and robustness, we propose a Multi-bit Feature Mixer technique (MFM) for fusing features of different bit-widths to enhance robustness across varying bit-widths. Moreover, we introduced the Group-wise Distillation Loss (GD-Loss) to enhance the correlation between different bit-width groups and further improve the overall performance of PTMQ. Extensive experiments demonstrate that PTMQ achieves comparable performance to existing state-of-the-art post-training quantization methods, while optimizing it speeds up by 100$\times$ compared to recent multi-bit quantization works. Code can be available at https://github.com/xuke225/PTMQ.

What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation

Quantization has emerged as a promising technique for improving the memory and computational efficiency of large language models (LLMs). Though the trade-off between performance and efficiency is well-known, there is still much to be learned about the relationship between quantization and LLM performance. To shed light on this relationship, we propose a new perspective on quantization, viewing it as perturbations added to the weights and activations of LLMs. We call this approach ``the lens of perturbation". Using this lens, we conduct experiments with various artificial perturbations to explore their impact on LLM performance. Our findings reveal several connections between the properties of perturbations and LLM performance, providing insights into the failure cases of uniform quantization and suggesting potential solutions to improve the robustness of LLM quantization. To demonstrate the significance of our findings, we implement a simple non-uniform quantization approach based on our insights. Our experiments show that this approach achieves minimal performance degradation on both 4-bit weight quantization and 8-bit quantization for weights and activations. These results validate the correctness of our approach and highlight its potential to improve the efficiency of LLMs without sacrificing performance.

Data-Driven Event-Triggered Control for Nonlinear Multi-Agent Systems With Uniform Quantization

Data-Driven Event-Triggered Control for Nonlinear Multi-Agent Systems With Uniform Quantization

Event-Triggered-Based Distributed Consensus Tracking for Nonlinear Multiagent Systems With Quantization.

In this article, an observer-based adaptive neural network (NN) event-triggered distributed consensus tracking problem is investigated for nonlinear multiagent systems with quantization. In the first place, the limited capacity of the communication channel between agents is considered. The event-trigger mechanism and dynamic uniform quantizers are set up to reduce information transmission. The next NN is utilized to handle the unknown nonlinear functions. Finally, in order to estimate the unmeasurable states, an NN-based state observer is designed for each agent by using a dynamic gain function. To settle the difficulty caused by the coupling effects of event-triggered conditions and the scaling function in dynamic uniform quantizers and observers, a distributed control protocol with estimated information of its neighbors is designed, which ensures distributed consensus tracking of the nonlinear multiagent systems without incurring the Zeno behavior. The effectiveness of the control protocol is illustrated by a simulation example.

Quantized iterative learning control of communication-constrained systems with encoding and decoding mechanism

In practical applications, due to the limited communication bandwidth, the network control systems (NCSs) are prone to data dropouts when the load is high. In this paper, the problem of quantized iterative learning control (ILC) based on encoding and decoding mechanism for such communication-constrained systems is studied. By combining the encoding and decoding mechanism with the uniform quantizer, the network burden and the impact of quantization error on the tracking performance of the systems are significantly mitigated. Meanwhile, data dropouts are represented as the Bernoulli random variable model, and an ILC law based on gradient is designed. When data dropouts occur, the signals maintain the value of the previous trial, otherwise the signals are updated. For this kind of learning framework, the asymptotic zero-error tracking performance has been rigorously proven for the uniform quantizer. To validate the proposed design, a joint motion of an industrial robot in the horizontal plane is simulated as an example.

Distributed Nash Equilibrium Seeking Strategies Under Quantized Communication

This paper is concerned with distributed Nash equilibrium seeking strategies under quantized communication. In the proposed seeking strategy, a projection operator is synthesized with a gradient search method to achieve the optimization of players' objective functions while restricting their actions within required non-empty, convex and compact domains. In addition, a leader-following consensus protocol, in which quantized information flows are utilized, is employed for information sharing among players. More specifically, logarithmic quantizers and uniform quantizers are investigated under both undirected and connected communication graphs and strongly connected digraphs, respectively. Through Lyapunov stability analysis, it is shown that players' actions can be steered to a neighborhood of the Nash equilibrium with logarithmic and uniform quantizers, and the quantified convergence error depends on the parameter of the quantizer for both undirected and directed cases. A numerical example is given to verify the theoretical results.

Finite-time adaptive prescribed performance DSC for pure feedback nonlinear systems with input quantization and unmodeled dynamics

<abstract><p>This paper presents a new prescribed performance-based finite-time adaptive tracking control scheme for a class of pure-feedback nonlinear systems with input quantization and dynamical uncertainties. To process the input signal, a new quantizer combining the advantages of a hysteresis quantizer and uniform quantizer has been used. Radial basis function neural networks have been utilized to approximate unknown nonlinear smooth functions. An auxiliary system has been employed to estimate unmodeled dynamics by producing a dynamic signal. By introducing a hyperbolic tangent function and performance function, the tracking error was made to fall within the prescribed time-varying constraints. Using modified dynamic surface control (DSC) technology and a finite-time control method, a novel finite-time controller has been designed, and the singularity problem of differentiating each virtual control scheme in the existing finite-time control scheme has been removed. Theoretical analysis shows that all signals in the closed-loop system are semi-globally practically finite-time stable, and that the tracking error converges to a prescribed time-varying region. Simulation results for two numerical examples have been provided to illustrate the validity of the proposed control method.</p></abstract>

Communication-efficient ADMM using quantization-aware Gaussian process regression

In networks consisting of agents communicating with a central coordinator and working together to solve a global optimization problem in a distributed manner, the agents are often required to solve private proximal minimization subproblems. Such a setting often requires a decomposition method to solve the global distributed problem, resulting in extensive communication overhead. In networks where communication is expensive, it is crucial to reduce the communication overhead of the distributed optimization scheme. Gaussian processes (GPs) are effective at learning the agents' local proximal operators, thereby reducing the communication between the agents and the coordinator. We propose combining this learning method with adaptive uniform quantization for a hybrid approach that can achieve further communication reduction. In our approach, due to data quantization, the GP algorithm is modified to account for the introduced quantization noise statistics. We further improve our approach by introducing an orthogonalization process to the quantizer's input to address the inherent correlation of the input components. We also use dithering to ensure uncorrelation between the quantizer's introduced noise and its input. We propose multiple measures to quantify the trade-off between the communication cost reduction and the optimization solution's accuracy/optimality. Under such metrics, our proposed algorithms can achieve significant communication reduction for distributed optimization with acceptable accuracy, even at low quantization resolutions. This result is demonstrated by simulations of a distributed sharing problem with quadratic cost functions for the agents.

Vina-FPGA-Cluster: Multi-FPGA Based Molecular Docking Tool with High-Accuracy and Multi-Level Parallelism.

AutoDock Vina (Vina) stands out among numerous molecular docking tools due to its precision and comparatively high speed, playing a key role in the drug discovery process. Hardware acceleration of Vina on FPGA platforms offers a high energy-efficiency approach to speed up the docking process. However, previous FPGA-based Vina accelerators exhibit several shortcomings: 1) Simple uniform quantization results in inevitable accuracy drop; 2) Due to Vina's complex computing process, the evaluation and optimization phase for hardware design becomes extended; 3) The iterative computations in Vina constrain the potential for further parallelization. 4) The system's scalability is limited by its unwieldy architecture. To address the above challenges, we propose Vina-FPGA-cluster, a multi-FPGA-based molecular docking tool enabling high-accuracy and multi-level parallel Vina acceleration. Standing upon the shoulders of Vina-FPGA, we first adapt hybrid fixed-point quantization to minimize accuracy loss. We then propose a SystemC-based model, accelerating the hardware accelerator architecture design evaluation. Next, we propose a novel bidirectional AG module for data-level parallelism. Finally, we optimize the system architecture for scalable deployment on multiple Xilinx ZCU104 boards, achieving task-level parallelism. Vina-FPGA-cluster is tested on three representative molecular docking datasets. The experiment results indicate that in the context of RMSD (for successful docking outcomes with metrics below 2Å), Vina-FPGA-cluster shows a mere 0.2% lose. Relative to CPU and Vina-FPGA, Vina-FPGA-cluster achieves 27.33× and 7.26× speedup, respectively. Notably, Vina-FPGA-cluster is able to deliver the 1.38× speedup as GPU implementation (Vina-GPU), with just the 28.99% power consumption.

Uniform Quantization for Multi-Antenna Amplify–Quantize–Forward Relay

Uniform Quantization for Multi-Antenna Amplify–Quantize–Forward Relay