Non-uniform Quantization Research Articles

A method of multi-bit optical coding by constructing Brewster angles to enhance photonic spin Hall effect

The photonic spin Hall effect, characterized by its polarization-driven spin-related displacement, has played an essential role in coding information processing. In this Letter, using a two-period double-layer dielectric plate, the displacement changes within the angle region are first observed, and a two-bit effective coding is achieved through non-uniform quantization of displacement values and coding. Afterward, by constructing Brewster angles (BAs), effective three-bit coding is achieved at the selected BA by dividing the angle domain. By changing the relative refractive index of the constituent materials (nB), effective coding is achieved at corresponding angles (45°–85°). By connecting two-bit and three-bit coding, a five-bit coding is fulfilled. By setting the last six invalid values, effective correspondence of 26 English letters is obtained. At the same time, the largest displacement peak itself is huge and varies accordingly with different values of nB. This work can provide some reference for designing optical coders.

Robust Symbol and Frequency Synchronization Method for Burst OFDM Systems in UAV Communication

This paper introduces a robust synchronization method for orthogonal frequency division multiplexing (OFDM) in multi-unmanned aerial vehicle (UAV) communication systems, focusing on minimizing overhead while achieving reliable synchronization. The proposed synchronization scheme enhances both frame efficiency and implementation simplicity. Initially, a high-efficiency frame structure is designed without a guard time interval, utilizing a preamble sequence to simultaneously achieve both symbol synchronization and automatic gain control (AGC) before demodulation. Subsequently, a novel 2-bit non-uniform quantization method for the Zadoff–Chu sequences is developed, enabling the correlation operations in the traditional symbol synchronization algorithm to be implemented via bitwise exclusive OR (XOR) and addition operations. The complexity of hardware implementation and the energy consumption for symbol synchronization can be reduced significantly. Furthermore, the impact of AGC on frequency synchronization performance is examined, and an improved frequency synchronization method based on AGC gain compensation is proposed. Finally, the performance of the proposed method is rigorously analyzed and compared with that of the traditional method through computer simulations, demonstrating the effectiveness and superiority of the proposed approach.

On robustification of digital event-based controllers for control-affine nonlinear systems

In this paper, the robust digital stabilization problem of nonlinear systems is investigated. In particular, a methodology for the design of robust quantized sampled-data stabilizers updated via an event-triggered mechanism is provided for time-varying control-affine nonlinear systems affected by actuation disturbances and measurement noises. The notion of time-varying steepest descent feedback (TSDF), continuous or not, and the Input-to-State Stability (ISS) redesign methodology are used for the development of the proposed robust event-based digital controller. Under the assumption that the actuation disturbances and measurement noises are bounded with a-priori known bounds and that the amplitude of the measurement noises satisfies a certain condition related to the new added robustification term, the following result is proved: there exist a suitably fast sampling and an accurate quantization of the input/output channels such that the digital implementation of robustified TSDF controllers, updated through a proposed event-triggered mechanism, ensures semi-global practical stability of the related closed-loop system regardless of the above uncertainties. In the methodology here proposed, time-varying sampling periods and the non-uniform quantization of the input/output channels are allowed. Moreover, the theory here developed includes the analysis of the intersampling system behaviour. Possible discontinuities in the function describing the TSDF at hand are also managed. The provided results are validated through a numerical example.

Adaptive non-iterative histogram-based hologram quantization

Quantization is widely used for compression, storage, transmission and display of computer-generated and digital holograms. It reduces number of hologram gradations to obtain better ratio of the quality of reconstructed 2D and 3D objects to the size of holographic images or videos. This paper proposes 2 methods of non-iterative quantization of holograms: adaptive non-uniform quantization based on histogram processing with peak shift (ANHS method) and additional application of logarithmic companding (ANHSL method). They were compared with 4 methods of iterative and non-iterative quantization on holograms with sizes up to 2048 × 2048 pixels. The proposed methods showed the best reconstruction quality that is higher up to 59 % in comparison with non-iterative uniform quantization. Especially the proposed methods give higher quality at little number of hologram gradations (from 2 to 16 gradations). In comparison with the most qualitative iterative methods (Lloyd-Max and k-means) the quality is increased up to 9 %, and processing time is reduced in 10–1000 times.

High-Throughput Polar Code Decoders with Information Bottleneck Quantization.

In digital baseband processing, the forward error correction (FEC) unit belongs to the most demanding components in terms of computational complexity and power consumption. Hence, efficient implementation of FEC decoders is crucial for next-generation mobile broadband standards and an ongoing research topic. Quantization has a significant impact on the decoder area, power consumption and throughput. Thus, lower bit widths are preferred for efficient implementations but degrade the error correction capability. To address this issue, a non-uniform quantization based on the Information Bottleneck (IB) method is proposed that enables a low bit width while maintaining the essential information. Many investigations on the use of the IB method for Low-density parity-check code) LDPC decoders exist and have shown its advantages from an implementation perspective. However, for polar code decoder implementations, there exists only one publication that is not based on the state-of-the-art Fast Simplified Successive-Cancellation (Fast-SSC) decoding algorithm, and only synthesis implementation results without energy estimation are shown. In contrast, our paper presents several optimized Fast-SSC polar code decoder implementations using IB-based quantization with placement and routing results using advanced 12 nm FinFET technology. Gains of up to 16% in area and 13% in energy efficiency are achieved with IB-based quantization at a Frame Error Rate (FER) of 10-7 and a polar code of N=1024,R=0.5 compared to state-of-the-art decoders.

What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation

Quantization has emerged as a promising technique for improving the memory and computational efficiency of large language models (LLMs). Though the trade-off between performance and efficiency is well-known, there is still much to be learned about the relationship between quantization and LLM performance. To shed light on this relationship, we propose a new perspective on quantization, viewing it as perturbations added to the weights and activations of LLMs. We call this approach ``the lens of perturbation". Using this lens, we conduct experiments with various artificial perturbations to explore their impact on LLM performance. Our findings reveal several connections between the properties of perturbations and LLM performance, providing insights into the failure cases of uniform quantization and suggesting potential solutions to improve the robustness of LLM quantization. To demonstrate the significance of our findings, we implement a simple non-uniform quantization approach based on our insights. Our experiments show that this approach achieves minimal performance degradation on both 4-bit weight quantization and 8-bit quantization for weights and activations. These results validate the correctness of our approach and highlight its potential to improve the efficiency of LLMs without sacrificing performance.

Latitude-Adaptive Integer Bit Allocation for Quantization of Omnidirectional Images

Omnidirectional images have gained significant popularity and drawn great attention nowadays, which poses challenges to omnidirectional image processing in solving the bottleneck of storage and transmission. Projecting onto a two-dimensional image plane is generally used to compress an omnidirectional image. However, the most commonly used projection format, the equirectangular projection (ERP), results in a significant amount of redundant samples in the polar areas, thus incurring extra bitrate and geometric distortion. We derive the optimal latitude-adaptive bit allocation for each image tile. Subsequently, we propose a greedy algorithm for non-negative integer bit allocation (NNIBA) for non-uniform quantization under an omnidirectional image quality metric WMSE. In our experiment, we design quantization tables based on JPEG and compare our approach with other sampling-related methods. Our method achieves an average bit saving of 7.9% compared with JPEG while outperforming other sampling-related methods. Besides, we compare our non-uniform quantization approach with two proposed bit allocation methods, achieving an average improvement of 0.35 dB and 2.66 dB under WS-PSNR, respectively. The visual quality assessment also confirms the superiority of our method.

NQFL: Nonuniform Quantization for Communication Efficient Federated Learning

NQFL: Nonuniform Quantization for Communication Efficient Federated Learning

On the design and the digital implementation of observer‐based controllers for tracking of nonlinear time‐delay systems

AbstractIn this article, the tracking control problem for a class of nonlinear time‐delay systems is investigated. In particular, a new methodology for the design and the digital implementation of observer‐based tracking controllers is provided for a class of control‐affine nonlinear systems with state delays. First, a procedure for the design of continuous‐time observer‐based tracking controllers ensuring the global asymptotic stability of the corresponding closed‐loop tracking error system is provided for the considered class of systems. Then, sufficient conditions are provided for the existence of a suitably fast sampling and of an accurate quantization of the input/output channels such that the digital implementation of the proposed continuous‐time observer‐based tracking controller ensures the semi‐global practical stability property of the related sampled‐data quantized closed‐loop tracking error system, with arbitrarily small final target ball of the origin. Moreover, it is shown that, in the special case of delay‐free nonlinear systems, the sufficient conditions provided for the digital implementation of the proposed continuous‐time observer‐based tracking controller can be strongly relaxed. In the theory here developed, time‐varying sampling periods and nonuniform quantization of the input/output channels are taken into account. The proposed results are validated through examples concerning a class of neural networks systems and a class of time‐delay systems including, as a special case, a delay‐free actuated inverted pendulum.

Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression.

Light fields capture 3D scene information by recording light rays emitted from a scene at various orientations. They offer a more immersive perception, compared with classic 2D images, but at the cost of huge data volumes. In this paper, we design a compact neural network representation for the light field compression task. In the same vein as the deep image prior, the neural network takes randomly initialized noise as input and is trained in a supervised manner in order to best reconstruct the target light field Sub-Aperture Images (SAIs). The network is composed of two types of complementary kernels: descriptive kernels (descriptors) that store scene description information learned during training, and modulatory kernels (modulators) that control the rendering of different SAIs from the queried perspectives. To further enhance compactness of the network meanwhile retain high quality of the decoded light field, we propose modulator allocation and apply kernel tensor decomposition techniques, followed by non-uniform quantization and lossless entropy coding. Extensive experiments demonstrate that our method outperforms other state-of-the-art (SOTA) methods by a significant margin in the light field compression task. Moreover, after adapting descriptors, the modulators learned from one light field can be transferred to new light fields for rendering dense views, showing the potential of the solution for view synthesis.

CLUT-CIM: A Capacitance Lookup Table-Based Analog Compute-in-Memory Macro With Signed-Channel Training and Weight Updating for Nonuniform Quantization

CLUT-CIM: A Capacitance Lookup Table-Based Analog Compute-in-Memory Macro With Signed-Channel Training and Weight Updating for Nonuniform Quantization

Non-uniform-digital-analog radio-over-fiber scheme for future fronthaul based on a low-complexity 4-fold symmetric non-uniform quantization method

Non-uniform-digital-analog radio-over-fiber scheme for future fronthaul based on a low-complexity 4-fold symmetric non-uniform quantization method

On the robustification of digital event-based stabilizers for nonlinear time-delay systems

In this paper, the robust stabilization problem by means of quantized sampled-data event-based (QSE) controllers is investigated for nonlinear systems affected by state delays and unknown disturbances. In particular, a methodology for the design of robust QSE stabilizers is provided for control-affine nonlinear systems affected by unknown actuation disturbances and unknown measurement errors. Firstly, the notion of Steepest Descent Feedback (SDF), continuous or not, is suitably revised in order to deal with the robustification of event-based controllers. Then, Input-to-State Stability (ISS) redesign methodologies are used to provide the robustification term which is added to the SDF at hand in order to arbitrarily attenuate the effects of unknown external disturbances affecting the considered control scheme. A spline approximation approach is used in order to cope with the problem of the possible non-availability in the buffer of suitable past values of the system state required for the correct application of the proposed robust QSE controller. It is proved that there exist a suitably fast sampling and an accurate quantization of the input/output channels such that: the robust QSE implementation of SDFs, continuous or not, ensures the semi-global practical stability of the related closed-loop system, regardless of the above disturbances, provided that the observation errors affects marginally the new added control term. The stabilization in the sample-and-hold sense theory is used as a tool to prove the results. The provided results include the case of non-uniform quantization of the input/output channels and the case of aperiodic sampling. Applications are presented in order to validate the results.

Bit-Weight Adjustment for Bridging Uniform and Non-Uniform Quantization to Build Efficient Image Classifiers

Network quantization, which strives to reduce the precision of model parameters and/or features, is one of the most efficient ways to accelerate model inference and reduce memory consumption, particularly for deep models when performing a variety of real-time vision tasks on edge platforms with constrained resources. Existing quantization approaches function well when using relatively high bit widths but suffer from a decline in accuracy at ultra-low precision. In this paper, we propose a bit-weight adjustment (BWA) module to bridge uniform and non-uniform quantization, successfully quantizing the model to ultra-low bit widths without bringing about noticeable performance degradation. Given uniformly quantized data, the BWA module adaptively transforms these data into non-uniformly quantized data by simply introducing trainable scaling factors. With the BWA module, we combine uniform and non-uniform quantization in a single network, allowing low-precision networks to benefit from both the hardware friendliness of uniform quantization and the high performance of non-uniform quantization. We optimize the proposed BWA module by directly minimizing the classification loss through end-to-end training. Numerous experiments on the ImageNet and CIFAR-10 datasets reveal that the proposed approach outperforms state-of-the-art approaches across various bit-width settings and can even produce low-precision quantized models that are competitive with their full-precision counterparts.

A Hybrid Weight Quantization Strategy for Memristive Neural Networks

Due to the ability to store data and process information, the memristor-based neuromorphic system has attracted extensive attention. Its efficient parallel computing approach allows it to implement neural networks in hardware. However, due to the limitation of the range of memristor conductance, it is difficult to represent high-precision weights in memristive neural network. During the off-chip learning, it is crucial to find an efficient weight quantization scheme and map it to the memristor array. Therefore, a hybrid weight quantization strategy that combines uniform and non-uniform quantization is proposed to overcome these problems. Specifically, the curve fitting of pulse modulation for tantalum oxide-based memristor is carried out, and the mapping rules of weights are proposed to simplify the process of reading verification. Furthermore, the hybrid quantization strategy is proposed and applied to a multilayer perceptron and a convolutional neural network, respectively. The effectiveness and robustness of the hybrid quantization scheme are verified in the MNIST dataset. Experiments show that the proposed hybrid quantization scheme can achieve 99.26% accuracy at 4 bits and tolerate 20% noise interference. The simulation results in this paper also provide an effective solution for the hardware implementation of memristive neural networks.

High Order QAM Modulation in Massive MIMO Systems With Asymmetrically Quantized 1-Bit ADCs

Deploying 1-bit analog-to-digital converters (ADCs) in massive multiple-input multiple-output (MIMO) systems is promising to reduce the energy consumption. However, serious quantization errors limit the feasibility of high order quadrature amplitude modulation (QAM). This paper focuses on the performance analysis of 1-bit ADC massive MIMO systems with high order QAM. Firstly, we theoretically analyze the relationship between the quantization error and various quantization parameters. We reveal that there exists an overlapping of constellations at high signal to noise ratio (SNR) with symmetric quantization. This is because the amplitude information in the detected signal at high SNR is lost which is multiplied by the zero quantization threshold in the symmetric quantization. To overcome this problem, asymmetric quantization should be considered. Based on the analysis of the quantization error, we propose two approaches to recover the amplitude information. One is a near unbiased detection (NUD) by scaling the traditional linear detector and optimizing the quantization threshold. The other is a sub-array detection based on non-uniform quantization (SAD-NQ) to correct the deviation of the detection result by averaging over sub-arrays. Simulation results show that the proposed approaches can significantly improve the detection performance of high order QAM signal in the 1-bit ADC massive MIMO systems.

High-Performance and Lightweight AI Model for Robot Vacuum Cleaners with Low Bitwidth Strong Non-Uniform Quantization

Artificial intelligence (AI) plays a critical role in the operation of robot vacuum cleaners, enabling them to intelligently navigate to clean and avoid indoor obstacles. Due to limited computational resources, manufacturers must balance performance and cost. This necessitates the development of lightweight AI models that can achieve high performance. Traditional uniform weight quantization assigns the same number of levels to all weights, regardless of their distribution or importance. Consequently, this lack of adaptability may lead to sub-optimal quantization results, as the quantization levels do not align with the statistical properties of the weights. To address this challenge, in this work, we propose a new technique called low bitwidth strong non-uniform quantization, which largely reduces the memory footprint of AI models while maintaining high accuracy. Our proposed non-uniform quantization method, as opposed to traditional uniform quantization, aims to align with the actual weight distribution of well-trained neural network models. The proposed quantization scheme builds upon the observation of weight distribution characteristics in AI models and aims to leverage this knowledge to enhance the efficiency of neural network implementations. Additionally, we adjust the input image size to reduce the computational and memory demands of AI models. The goal is to identify an appropriate image size and its corresponding AI models that can be used in resource-constrained robot vacuum cleaners while still achieving acceptable accuracy on the object classification task. Experimental results indicate that when compared to the state-of-the-art AI models in the literature, the proposed AI model achieves a 2-fold decrease in memory usage from 15.51 MB down to 7.68 MB while maintaining the same accuracy of around 93%. In addition, the proposed non-uniform quantization model reduces memory usage by 20 times (from 15.51 MB down to 0.78 MB) with a slight accuracy drop of 3.11% (the classification accuracy is still above 90%). Thus, our proposed high-performance and lightweight AI model strikes an excellent balance between model complexity, classification accuracy, and computational resources for robot vacuum cleaners.

The reversibility of cancelable biometric templates based on iterative perturbation stochastic approximation strategy

Researchers have been proposing different matching frameworks for cancelable template design to improve the matching performance and security of biometric authentication systems. Despite the advantages provided by these systems, they are often vulnerable to different attacks due to their templates' invertible nature. This study utilized an iterative perturbation stochastic approximation procedure to analyze the irreversible order-based encoding approach used in cancelable biometric template transformation. The strategy begins by exploiting the scores corresponding to the encoded words, then a mixed variable-based iterative perturbation is used to generate independent random elements and consequently the corresponding template from the scores. The strategy exploits the vulnerability of three different cancelable systems it has tested, demonstrating that other techniques of similar nature fall short of the security standard for attack mitigation that cancelable biometric systems require. After analyzing the reasons that exploit the system's vulnerability, we use a parameterized thresholding non-uniform quantization as a countermeasure to boost the system's robustness while maintaining a balanced performance-security trade-off. As a result, the system can evade attacks without significantly hindering its matching performance. Finally, generality and computational complexity evaluations on different modalities and strategies validate the attack's efficacy and realism, respectively.

Intermediate deep feature coding for human–machine vision collaboration

Traditional image coding are mainly designed for human vision. While for collaborative intelligence, deep feature coding is specific for machine vision, which includes feature extraction and compression. Actually, deep features can build a bridge between human and machine vision. Therefore, we focus on generalized deep feature extraction and compression for multitask, which includes image reconstruction task for human vision and computer visual tasks for machine vision. After analyzing correlation among multitask, a reconstruction guided feature extraction strategy and feature fusion based network are proposed to get more generalized intermediate deep feature, which contains sufficient information friendly for human and machine vision. Besides, a non-uniform quantization method based on importance and a compact representation method for feature distribution information protection are proposed for high efficiency feature coding. Eventually, we come up with an entire intermediate deep feature coding framework including feature extraction and compression. Experimental results indicate the performance gains with our framework.

Characterizing Coherent Integrated Photonic Neural Networks Under Imperfections

Integrated photonic neural networks (IPNNs) are emerging as promising successors to conventional electronic AI accelerators as they offer substantial improvements in computing speed and energy efficiency. In particular, coherent IPNNs use arrays of Mach–Zehnder interferometers (MZIs) for unitary transformations to perform energy-efficient matrix-vector multiplication. However, the underlying MZI devices in IPNNs are susceptible to uncertainties stemming from optical lithographic variations and thermal crosstalk and can experience imprecisions due to non-uniform MZI insertion loss and quantization errors due to low-precision encoding in the tuned phase angles. In this article, we, for the first time, systematically characterize the impact of such uncertainties and imprecisions (together referred to as imperfections) in IPNNs using a bottom-up approach. We show that their impact on IPNN accuracy can vary widely based on the tuned parameters (e.g., phase angles) of the affected components, their physical location, and the nature and distribution of the imperfections. To improve reliability measures, we identify critical IPNN building blocks that, under imperfections, can lead to catastrophic degradation in the classification accuracy. We show that under multiple simultaneous imperfections, the IPNN inferencing accuracy can degrade by up to 46%, even when the imperfection parameters are restricted within a small range. Our results also indicate that the inferencing accuracy is sensitive to imperfections affecting the MZIs in the linear layers next to the input layer of the IPNN.