Non-uniform Quantization Research Articles

Reconstruction-Computation-Quantization (RCQ): A Paradigm for Low Bit Width LDPC Decoding

This paper uses the reconstruction-computation-quantization (RCQ)paradigm to decode low-density parity-check (LDPC) codes. RCQ facilitates dynamic non-uniform quantization to achieve good frame error rate (FER) performance with very low message precision. For message-passing according to a flooding schedule, the RCQ parameters are designed by discrete density evolution. Simulation results on an IEEE 802.11 LDPC code show that for 4-bit messages, a flooding Min Sum RCQ decoder outperforms table-lookup approaches such as information bottleneck (IB) or Min-IB decoding, with significantly fewer parameters to be stored. Additionally, this paper introduces layer-specific RCQ, an extension of RCQ decoding for layered architectures. Layer-specific RCQ uses layer-specific message representations to achieve the best possible FER performance. For layer-specific RCQ, this paper proposes using layered discrete density evolution featuring hierarchical dynamic quantization (HDQ) to design parameters efficiently. Finally, this paper studies field-programmable gate array (FPGA) implementations of RCQ decoders. Simulation results for a (9472, 8192) quasi-cyclic (QC) LDPC code show that a layered Min Sum RCQ decoder with 3-bit messages achieves more than a 10% reduction in LUTs and routed nets and more than a 6% decrease in register usage while maintaining comparable decoding performance, compared to a 5-bit offset Min Sum decoder.

Algorithm/architecture solutions to improve beyond uniform quantization in embedded DNN accelerators

The choice of data type has a major impact on speed, accuracy, and power consumption of deep learning accelerators. Quantizing the weights and activations of neural networks to integer based computation is an industry standard for reducing memory footprint and computation cost of inference in embedded systems. Uniform weight quantization can be used for tasks where accuracy drop can be tolerated. However, the drop in accuracy due to a uniform quantization might be non-negligible especially when performed on shallow networks, complex computer vision tasks, or with lower-bit integers.In this paper, we introduce a software and a hardware solution to improve on a baseline integer based uniform quantization so that it can be run on lower power systems and with even less bits. We also introduce a novel encoding technique on top of our software solution specific for partial sums to significantly reduce memory footprint, latency and energy consumption due to movement of partial sums. The proposed SW solution exploits non-uniform piece-wise linear quantization to improve accuracy by capturing bell shaped distribution of weights while still using INT-based computation units. The proposed partial sum encoding can be applied to the partial sums regardless of uniform or non-uniform quantization. The proposed HW solution can either combine integers to make larger integers or turn them into Floating-Point operations so that various levels can have various precisions or data types, if necessary. To do so, we studied upper limits of precision we need in our compute units to support floating point inner product operations. It turns out that we can improve upon integer IPUs to perform accurate floating-point operations without introducing large shift units or wide adder trees.Our proposed SW solution (PWLQ) achieves the state-of-the-art results on all cases and it outperforms all other methods with a large margin. The proposed partial sum encoding technique effectively compresses the partial sum of networks like Resnet-50 down to 12bits (from Int64/32) without loss in accuracy. The proposed HW architecture achieves area improvements of up to 46% in TOPS/mm2 with power efficiency improvements of up to 63% in TOPS/W when compared to state of the art mixed precision implementation.

Signature of nonuniform area quantization on black hole echoes

A classical black hole is characterized by a horizon that absorbs the radiation of all frequencies incident on it. The perturbation of these black holes is well understood via exponentially damped sinusoids known as quasinormal modes. Any departure from such classical behavior near the horizon may induce significant modifications in the late-time evolution of the perturbation leading to so-called gravitational wave echoes. This work considers the effect of black hole area quantization on the formation of gravitational wave echoes. We investigate how the resulting echo waveform may depend on various model parameters. Our study opens up a new window to distinguish different models of area quantization using future gravitational wave observations and provides a novel probe to study the near-horizon physics.

Redundancy cancellation of compressed measurements by QRS complex alignment.

The demand for long-term continuous care has led healthcare experts to focus on development challenges. On-chip energy consumption as a key challenge can be addressed by data reduction techniques. In this paper, the pseudo periodic nature of ElectroCardioGram(ECG) signals has been used to completely remove redundancy from frames. Compressing aligned QRS complexes by Compressed Sensing (CS), result in highly redundant measurement vectors. By removing this redundancy, a high cluster of near zero samples is gained. The efficiency of the proposed algorithm is assessed using the standard MIT-BIH database. The results indicate that by aligning ECG frames, the proposed technique can achieve superior reconstruction quality compared to state-of-the-art techniques for all compression ratios. This study proves that by aligning ECG frames with a 0.05% unaligned frame rate(R-peak detection error), more compression could be gained for PRD > 5% when 5-bit non-uniform quantizer is used. Furthermore, analysis done on power consumption of the proposed technique, indicates that a very good recovery performance can be gained by only consuming 4.9μW more energy per frame compared to traditional CS.

Optimization of Quantized Analog Signal Processing Using Genetic Algorithms and μ-Law

Digital mismatch calibration for quantized analog (QA) signal processing is proposed for the first time. Since the proposed calibration mechanism does not require uniform QA slicer levels, non-uniform quantization can be applied to improve the system performance. We propose two methods utilizing the genetic algorithm and <inline-formula> <tex-math notation="LaTeX">$\mu $ </tex-math></inline-formula>-law to find non-uniform slicer levels offering superior performance compared to uniform levels. Simulations show that for a QA amplifier consisting of 32 slices, the signal-to-noise-and-distortion ratio (SNDR) under a multitone input can be doubled by adjusting only the quantization levels while maintaining the same structure and same power, compared to uniform quantization levels that provide 54 dB of SNDR.

Graph-Based Depth Denoising & Dequantization for Point Cloud Enhancement.

A 3D point cloud is typically constructed from depth measurements acquired by sensors at one or more viewpoints. The measurements suffer from both quantization and noise corruption. To improve quality, previous works denoise a point cloud a posteriori after projecting the imperfect depth data onto 3D space. Instead, we enhance depth measurements directly on the sensed images a priori, before synthesizing a 3D point cloud. By enhancing near the physical sensing process, we tailor our optimization to our depth formation model before subsequent processing steps that obscure measurement errors. Specifically, we model depth formation as a combined process of signal-dependent noise addition and non-uniform log-based quantization. The designed model is validated (with parameters fitted) using collected empirical data from a representative depth sensor. To enhance each pixel row in a depth image, we first encode intra-view similarities between available row pixels as edge weights via feature graph learning. We next establish inter-view similarities with another rectified depth image via viewpoint mapping and sparse linear interpolation. This leads to a maximum a posteriori (MAP) graph filtering objective that is convex and differentiable. We minimize the objective efficiently using accelerated gradient descent (AGD), where the optimal step size is approximated via Gershgorin circle theorem (GCT). Experiments show that our method significantly outperformed recent point cloud denoising schemes and state-of-the-art image denoising schemes in two established point cloud quality metrics.

Feature Compensation Network Based on Non-Uniform Quantization of Channels for Digital Image Manipulation Forensics

Feature Compensation Network Based on Non-Uniform Quantization of Channels for Digital Image Manipulation Forensics

Robust Quantized Sampled–Data Stabilization for a Class of Lipschitz Nonlinear Systems With Time–Varying Uncertainties

In this letter, a robust quantized sampled-data controller is provided for a class of nonlinear systems affected by time-varying uncertainties, actuation disturbances and measurement noises. Sufficient conditions based on linear matrix inequalities and ensuring the existence of the proposed robust quantized sampled-data controller are given. Quantization of both state measurements and input signals is simultaneously considered. Input-to-state stability redesign technique is used in order to attenuate the effects of bounded actuation disturbances and of bounded observation errors. It is proved that, under suitably fast sampling and accurate quantization of the input/output channels, the proposed controller achieves the semi-global practical stability, with arbitrarily small final target ball, of the related quantized sampled-data closed-loop system provided that the observation errors do not affect (or affect marginally) the robustification term added in the controller and, that the bounds of the actuation disturbances as well as of the observation errors are a priori known. The theory here developed includes also the cases of time-varying sampling intervals and of non-uniform quantization of the input/output channels as well as the stability analysis of the inter-sampling system behavior. The provided results are validated through an example of one-link manipulator.

Novel Low Complexity BP Decoding Algorithms for Polar Codes: Simplifying on Non-Linear Operations

The parallel nature of the belief propagation (BP) decoding algorithm for polar codes opens up a real possibility of high throughput and low decoding latency during hardware implementation. To address the problem that the BP decoding algorithm introduces high-complexity non-linear operations in the iterative messages update process, this paper proposes to simplify these operations and develops two novel low complexity BP decoding algorithms, namely, exponential BP (Exp-BP) decoding algorithm and quantization function BP (QF-BP) decoding algorithm. The proposed algorithms simplify the compound hyperbolic tangent function by using probability distribution fitting techniques. Specifically, the Exp-BP algorithm simplifies two types of non-linear operations into single non-linear operation using the piece-wise exponential model function, which can approximate the hyperbolic tangent function in the updating formula. The QF-BP algorithm eliminates non-linear operations using the non-uniform quantization in the updating formula, which is effective in reducing computational complexity. According to the simulation results, the proposed algorithms can reduce the computational complexity up to 50% in each iteration with a loss of less than 0.1 dB compared with the BP decoding algorithm, which can facilitate the hardware implementation.

3D image compression during ultrasound phased diagnostics based on wavelet subband coding planar scans

The proposed compression method is based on the application of a two-dimensional discrete fast wavelet transform (FWT) to planar scans of 3D ultrasound images in order to simultaneously reduce redundancy and suppress speckle at a fixed quota of bits. The main idea of the method is to fuse three rules for threshold processing the wavelet coefficients of the scans, uniform and non-uniform quantizers, and bit quota distributions over subbands of the scan FWT based on the proposed cost function. The simulation results have shown that at the encoding rate of up to 1 bit/pixel, the quantity of artefacts were decreased up to 5-7 % of the original quantity under a signal-to-speckle ratio more than 16 dB, and the structural similarity index (SSIM) increased to 0.94-0.97 for defects of rectangular, triangular and oval shapes. The paper also presents the results proving the effectiveness of the proposed method in comparison with some variants of the solution according to the scheme “pre-filtering + codec”.

Signature of nonuniform area quantization on gravitational waves

Quantum aspects of black holes may have observational imprints on their absorption and emission spectrum. In this work, we consider the possibility of non-uniform area quantization and its effects on the phasing of gravitational waveform from coalescing black hole inspirals. These observations may provide detectable effects distinct from that of a uniform area quantization and allow us to put bounds on various parameters of the underlying model. Our work can also be regarded as a novel test for the area-entropy proportionality of black hole solutions in general relativity.

TIMAQ: A Time-Domain Computing-in-Memory-Based Processor Using Predictable Decomposed Convolution for Arbitrary Quantized DNNs

Energy-efficient processors are crucial for accelerating deep neural networks (DNNs) on edge devices with limited battery capacity. To reduce energy consumption, time-domain computing-in-memory (TD-CIM) is a splendid architecture, which consumes low computation and memory access energy due to low toggle rate of time-based signals and less data movements, respectively. When deploying DNNs in TD-CIMs, quantization is required, which has two types: uniform quantization (UQ) and nonuniform quantization (NUQ). To reach the same accuracy for one DNN, NUQ achieves smaller model size than UQ. Due to varying weight distributions across layers, mixed-precision quantization can further reduce model size, without degrading accuracy. However, previous TD-CIMs are inefficient for mixed-precision NUQ-DNNs due to their adopted bit-serial convolution increasing computation amount significantly. To address that, we propose a unique-weight convolution to accelerate mixed-precision NUQ-DNNs by a special kernel decomposition, reducing computation count remarkably. Based on that, we design a TD-CIM-based processor, TIMAQ, with three architectural techniques: 1) bit-cross-flipping-based kernel decomposer to reduce memory accesses and operations of decomposing kernels; 2) dual-mode-complementary predictor to remove redundant computations; and 3) activation-weight-adaptive pulse quantizer to decrease pulse quantization energy and error. Fabricated in 28-nm CMOS technology and tested on 1–8-b NUQ-DNNs, TIMAQ achieves 2.4–152.7-TOPS/W peak energy efficiency.

Fastening the initial access in 5G NR sidelink for 6G V2X networks

The ever-increasing demand for intelligent, automated, and connected mobility solutions pushes for the development of innovative vehicular communications technology in the framework of the sixth Generation (6G) of cellular networks. A radical transformation on the physical layer is planned, with a paradigm shift towards beam-based millimeter Waves or sub-Terahertz communications, which require precise beam pointing for guaranteeing the communication link, especially in high mobility. A key design aspect is a fast and proactive Initial Access (IA) algorithm to select the optimal beam to be used. In this work, we investigate alternative IA techniques to fasten the current fifth-generation (5G) standard approach, targeting an efficient 6G design. First, we discuss cooperative position-based schemes that rely on the position information. Then, motivated by the intuition of a non-uniform distribution of the communication directions due to road topology constraints, we design two Probabilistic Codebook (PCB) techniques of prioritized beams. In the first one, the PCBs are built leveraging historical traffic information, while in the second one, we use the Hough Transform over the digital map to extract dominant road directions. We also show that the information coming from the angular probability distribution allows designing non-uniform codebook quantization, reducing the degradation of the performances compared to uniform one. Numerical simulation on realistic scenarios shows that PCB-based beam selection outperforms the 5G standard in terms of the number of IA trials, with a performance comparable to position-based methods, without requiring the signaling of sensitive information.

Distributed Event-Triggered Circle Formation Control for Multiagent Systems with Nonuniform Quantization

This article focuses on circle formation control problem of multiagent systems based on event-triggered strategy under limited communication bandwidth. In such system, each agent can only perceive the angular distance of its nearest neighbor in the counterclockwise direction, and the angular distance of the nearest neighbor in the clockwise direction needs to be obtained by communicating with each other. In order to address the aforementioned problem, a novel distributed algorithm based on the combination of nonuniform quantitative communication technology and event-triggered control is proposed. Sufficient conditions on circle formation control are derived under which the states of all agents can be confirmed to converge to some desired equilibrium point. Different from the traditional uniform quantization communication framework, nonuniform quantization can be beneficial for handling small signals and improving the performance of multiagent systems concerned. Furthermore, under the proposed policy, all the designed quantizers do not emerge saturated. Numerical simulation results are provided to verify the effectiveness of the proposed algorithm.

A Low-Complexity Decoder With Non-Uniform Quantization for Recursive BMST Codes

Recursive block Markov superposition transmission (rBMST) can be used to construct high-performance spatially coupled codes. When used in power-limited scenarios, low-complexity quantized decoders are required for the rBMST codes. In this letter, we introduce an efficient decoder for rBMST codes by incorporating non-uniform quantization (NUQ) into the threshold attenuated min-sum algorithm (TAMSA). The resulting decoder is referred to as the NUQ-TAMSA. We use the hierarchical dynamic quantization algorithm to design the non-uniform quantizer. To show the effectiveness of the NUQ-TAMSA, we carry out comprehensive comparisons. Our results show that, for low-resolution quantization, performance gains as large as 0.3 dB can be obtained with the NUQ-TAMSA. Numerical results also show that, for a wide range of coding rates, 5-bits non-uniform quantization is sufficient for NUQ-TAMSA to approach the performances of the floating-point based TAMSA.

Average Contiguous Duration—A Novel Metric for Characterizing Wireless Fading Channels

We define the average contiguous duration (ACD) of the received fading signal as the average time duration during which the signal envelope contiguously remains within a bounded amplitude interval. Also, the multi-interval ACD or L-ACD function is defined as the set of ACD values for L non-overlapping amplitude-intervals whose union spans the received envelope's amplitude range. We derive closed-form expressions of the ACD for Rayleigh, Rice, and Nakagami- m fading signals, which are widely analyzed in the literature. The derived expressions hold practical significance for several signal processing applications such as non-uniform quantization of channel samples for secret key generation (SKG) in physical layer security (PLS) techniques. The well-known channel fading metric of average fade duration (AFD) is shown as a special case of the proposed ACD metric and the proposed theoretical analysis is validated by simulations.

HEMP: High-order entropy minimization for neural network compression

We formulate the entropy of a quantized artificial neural network as a differentiable function that can be plugged as a regularization term into the cost function minimized by gradient descent. Our formulation scales efficiently beyond the first order and is agnostic of the quantization scheme. The network can then be trained to minimize the entropy of the quantized parameters, so that they can be optimally compressed via entropy coding. We experiment with our entropy formulation at quantizing and compressing well-known network architectures over multiple datasets. Our approach compares favorably over similar methods, enjoying the benefits of higher order entropy estimate, showing flexibility towards non-uniform quantization (we use Lloyd-max quantization), scalability towards any entropy order to be minimized and efficiency in terms of compression. We show that HEMP is able to work in synergy with other approaches aiming at pruning or quantizing the model itself, delivering significant benefits in terms of storage size compressibility without harming the model’s performance.

High-Speed Digital Detector for the Internet of Things Assisted by Signal’s Intensity Quantification

This paper proposes a high-speed digital detector for the Internet of Things (IoT) assisted by signal’s intensity quantification. The detector quantifies the amplitude of each pixel of the detected image and converts it into a digital signal, which can be directly applied to the IoT with wireless communication system. Two types of amplitude quantization algorithms, uniform quantization and non-uniform quantization, are applied to the detector, which further improves the quality of the detected image and the robustness of the image signal in a noisy environment. Related simulations have been established to verify the accuracy of the models and algorithms.

On Quantization for Secret Key Generation From Wireless Channel Samples

Physical layer security (PLS) techniques hold promise for augmenting secure communications in the 5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> generation of mobile wireless networks. Secret key generation (SKG) is a PLS technique which exploits the wireless propagation channel's randomness to generate symmetric key bits for information encryption and decryption. This work proposes symmetric key generation based on non-uniform quantization of the received signal strength (RSS) samples of a Nakagami-$m$ fading channel. The proposed strategy for non-uniform quantization for SKG aims to set quantization thresholds for maximal key randomness and high values of key generation rate (KGR) and key agreement probability (KAP). Finally, a framework is proposed to use single node RSS measurements, readily available in the literature, to generate RSS samples at the other link end. This framework facilitates the testing of new SKG algorithms that require simultaneous RSS measurements by the legitimate nodes, which are not readily available in the open literature. The effectiveness of the proposed SKG scheme is validated through Monte Carlo methods and the National Institute of Standards and Technology (NIST) test suite for assessing the randomness of the generated key sequence.

Average Contiguous Duration (ACD)-Based Quantization for Secret Key Generation in Generalized Gamma Fading Channels

The wireless channel-based Secret Key Generation (SKG) algorithms aim at securing the wireless link against unauthorized eavesdropping by exploiting the channel’s randomness for generating matching secret keys at the legitimate nodes for message encryption/decryption. To counter differences in hardware and noise conditions at the legitimate nodes, which can lead to key mismatch, the SKG algorithms typically include the intermediate steps of sampling, quantization, information reconciliation, and privacy amplification. These steps collectively aim to improve the performance trade-offs between Key Generation Rate (KGR), Key Agreement Probability (KAP), and Secret Key Randomness (SKR) properties. This paper derives a closed-form expression for the Average Contiguous Duration (ACD) of Generalized Gamma (GG) fading wireless channels. The ACD is a recently introduced novel quantifier for characterizing the second-order statistics of fading channels, which includes Average Fade Duration (AFD) as its special case. The proposed GG fading ACD expression is shown to include, as its special cases, the ACD for commonly observed fading distributions such as Gamma, Nakagami- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> , and Rayleigh. By exploiting the derived GG ACD expression, a multi-level quantization scheme for SKG is proposed that determines suitable quantization intervals for identical likelihood of an equal number of consecutive channel samples falling in each quantization interval. A comprehensive comparative analysis of the proposed ACD-based quantization for SKG is conducted in relation to conventional Uniform Quantization (UQ) and Cumulative Distribution Function (CDF)-based Non-Uniform Quantization (NUQ) schemes. The presented numerical results confirm the superior performance trade-off between KGR and KAP offered by the proposed ACD-based quantization in relation to that offered by UQ and CDF-based NUQ.