Scalar Quantization Research Articles

No-reference Bitstream-based Perceptual Quality Assessment of Octree-Lifting Encoded 3D Point Clouds.

No-reference point cloud quality assessment (PCQA) based on bitstreams uses information extracted from the bitstream for quality monitoring at network nodes. We develop a no-reference PCQA model based on bitstreams for the perceived quality assessment of Octree-Lifting coded point clouds. At first, our research explores the essential correlation between subjective visual quality degradation and the texture quantization parameter (TQP) when using lossless geometric coding. Then, we enhance the proposed model by incorporating texture complexity (TC) while taking into account the dependence of perceptual coding distortion on the texture characteristics of a point cloud. We estimate TC by utilizing TQP and calculating the average standard deviation of the Y-component of the attribute value ( Y_ std), both of which are extracted from the bitstream. Then, a texture distortion assessment model is constructed based on TQP and Y_ std. The integration of the texture distortion model with the position quantization scale (PQS) results in the derivation of an overall no-reference bitstream-based PCQA model, named streamPCQ-OL. The findings from the conducted experiments highlight a significant superiority of the proposed model over existing approaches in terms of performance. The dataset and source code will be publicly released and made available for access at https://github.com/qdushl/Waterloo-Point-Cloud-Database-4.0.

Adaptive control strategy of VSG parameters based on improved fuzzy control

Abstract A virtual synchronous generator (VSG) enables the converter output to have similar external characteristics to the synchronous generator, but its virtual rotary inertia and damping factor need to be flexibly adapted to strengthen the dynamic stability of the system. Given the shortcomings of existing adaptive strategies, this paper presents a VSG parameter adaptive control strategy based on improved fuzzy control. GRO is used to optimize the quantization factor and scale factor of the fuzzy controller, to strengthen the adaptive ability of the fuzzy controller to the rotary inertia and damping factor. Simulink simulation is used to verify the superiority of the control strategy.

A comprehensive optimization method for dual redundancy aviation electric propulsion control system

Abstract This paper comprehensively optimizes and improves the traditional vector control method to improve the control performance of dual-redundancy electric aircraft and ensure the stability of its system operation in complex meteorological environments. First, the PI controller of the speed loop is replaced with the designed fuzzy PI controller, and its quantization factor and scale factor parameters are adjusted using a beetle antennae search algorithm (BAS) to enhance the system’s ability to respond quickly and resist interference. Secondly, a sliding mode load observer was designed to feed forward the observed values to the speed loop output, solving the torque and current ripple problems caused by the direct input of the redundant electric propulsion system when the aircraft switches from a single-redundant cruise state to a dual-redundant high-speed state. Finally, the design method was simulated and validated using MATLAB/SIMULINK, verifying the feasibility of the design.

Research on the Autonomous Control Technology Used in the Slurry Mixing System of Cementing Units

Cementing is a critical link in oil and gas exploitation, in which slurry density control is particularly important. In this study, we examined a slurry mixing control system in order to solve the problem of time delays in the mixing system. The model of a slurry mixing system was built in accordance with the system’s structure. A Smith fuzzy PID (proportion integration differentiation) composite control solution is proposed herein, and the simulation results show that the adjustment time and overshoot are lower than those of the conventional PID control and Smith predictive compensation control. A genetic algorithm is utilized to optimize the quantization factor and scale factor of the Smith fuzzy PID controller. Following optimization, the rise time of the controller was found to be 0.45 s, which represents a decrease of 35.9%, the overshoot was reduced by 0.4%, and the stabilization time was reduced by 36.6%. Afterward, we built a cementing slurry mixing simulation experimental platform, and experiments were used to verify the feasibility and superiority of the Smith fuzzy PID controller optimized by the genetic algorithm in comparison with the conventional controllers. The study results thus provide a scientific basis for the engineering application of the autonomous control technology of the slurry mixing system in cementing units.

Post-training quantization for re-parameterization via coarse & fine weight splitting

Although neural networks have made remarkable advancements in various applications, they require substantial computational and memory resources. Network quantization is a powerful technique to compress neural networks, allowing for more efficient and scalable AI deployments. Recently, Re-parameterization has emerged as a promising technique to enhance model performance while simultaneously alleviating the computational burden in various computer vision tasks. However, the accuracy drops significantly when applying quantization on the re-parameterized networks. We identify that the primary challenge arises from the large variation in weight distribution across the original branches. To address this issue, we propose a coarse & fine weight splitting (CFWS) method to reduce quantization error of weight, and develop an improved KL metric to determine optimal quantization scales for activation. To the best of our knowledge, our approach is the first work that enables post-training quantization applicable on re-parameterized networks. For example, the quantized RepVGG-A1 model exhibits a mere 0.3% accuracy loss. The code is in https://github.com/NeonHo/Coarse-Fine-Weight-Split.git

Neural Speech Coding for Real-Time Communications Using Constant Bitrate Scalar Quantization

Neural Speech Coding for Real-Time Communications Using Constant Bitrate Scalar Quantization

Distributed soft video transmission based on hybrid digital and analog scheme

This study presents a novel distributed soft video delivery scheme using hybrid digital and analog framework, which realizes a relatively lightweight encoder as well as good robustness. Specifically, after applying scalar quantization to the interframes, the scaled analog information (quantization error) and the scaled digital information (quantized source) are superimposed and then transmitted. In this way, the proposed scheme directly delivers pseudo-analog symbols over the orthogonal frequency division multiplexing (OFDM) channels and involves no complicated digital codec. Moreover, we utilize cross-frame correlation to implement a distributed paradigm which further reduces the encoder complexity since the complex motion estimation and compensation algorithms are transferred to the decoder. Accordingly, the power distortion optimization scheme, which resolve the allocation of power and the parameters optimization to achieve minimum transmission distortion, is proposed. To solve it, first, we formulate the power distortion expressions regarding quantization parameters and power allocation coefficients. Subsequently, we divide the problem into two sub-problems based on the fast coordinate descent method and further propose a greedy iterative algorithm to optimize them. We also develop a data-driven optimization algorithm based on deep learning that reduces the additional delay brought by the iterative optimization method. At the receiver, we estimate the quantization output and quantization error by the modified linear least squares estimation with the virtual noise variance. Based on the simulation results, the proposed framework has a better performance in terms of peak signal-to-noise ratio and structural similarity than the relevant cutting-edge schemes while maintaining good robustness.

Image Coding based Orthogonal Polynomials Multiresolution Analysis with Joint Probability Context Modeling and Modified Golomb-Rice Entropy Coding

The work proposes, a JPEG2000 like compression technique which is based on multiresolution analysis of orthogonal polynomials transformation (OPT) coefficients has been presented with bit modeling for Golomb-Rice entropy coding. Initially, the image under analysis is divided into blocks and OPT is applied to each divided blocks. Then, transformed coefficients are represented as sub bands like structure (multiresolution) and scalar quantization is carried out to the transformed coefficients to reduce the precision. The quantized coefficients are then bit modelled in the bit plane using a joint probability statistical model, and significant bits in the bit plane are chosen. On the selected relevant bits, a geometrically distributed set of context is modelled for further encoding with modified Golomb-Rice encoding to provide compressed data. The decompression procedure is just the reverse of compression procedure. Experiments and analysis are carried out to demonstrate the efficiency of the proposed compression scheme in terms of compression ratio and Peak-Signal-to Noise Ratio (PSNR), and the results are encouraging

Overflow-free Compute Memories for Edge AI Acceleration

Compute memories are memory arrays augmented with dedicated logic to support arithmetic. They support the efficient execution of data-centric computing patterns, such as those characterizing Artificial Intelligence (AI) algorithms. These architectures can provide computing capabilities as part of the memory array structures (In-Memory Computing, IMC) or at their immediate periphery (Near-Memory Computing, NMC). By bringing the processing elements inside (or very close to) storage, compute memories minimize the cost of data access. Moreover, highly parallel (and, hence, high-performance) computations are enabled by exploiting the regular structure of memory arrays. However, the regular layout of memory elements also constrains the data range of inputs and outputs, since the bitwidths of operands and results stored at each address cannot be freely varied. Addressing this challenge, we herein propose a HW/SW co-design methodology combining careful per-layer quantization and inter-layer scaling with lightweight hardware support for overflow-free computation of dot-vector operations. We demonstrate their use to implement the convolutional and fully connected layers of AI models. We embody our strategy in two implementations, based on IMC and NMC, respectively. Experimental results highlight that an area overhead of only 10.5% (for IMC) and 12.9% (for NMC) is required when interfacing with a 2KB subarray. Furthermore, inferences on benchmark CNNs show negligible accuracy degradation due to quantization for equivalent floating-point implementations.

An Efficient Electrocardiography Data Compression

In healthcare, electrocardiography (ECG) sensors generate a large amount of heart electrical signal that must be efficiently compressed to enable fast data transfer and reduce storage costs. Existing methods for ECG data compression do not fully exploit the characteristics of ECG signals, leading to suboptimal compression. This study proposes a data compression technique for ECG data by exploiting the known characteristics of ECG signals. Our approach combines Savitzky-Golay filtering, detrending, discrete cosine transform, scalar quantization, run-length encoding, and Huffman coding for the effective compression. To optimize the compression performance, we generated quantization intervals tailored to the ECG data characteristics. The proposed method experimentally produces a high compression ratio of 127.61 for a design parameter K = 8, a minimum percentage root mean square difference of 1.03% for K = 128, and a maximum quality score (QS) of 39.78, where K is the number of quantization intervals. Moreover, we compared the proposed method to state-of-the-art methods on a widely used ECG benchmark dataset. We found that the proposed method outperforms the others in terms of the QS, which measures the overall compression-decompression ability. By enabling more storage and faster data transfer, the proposed method can facilitate the widespread use and analysis of large volumes of ECG data, thereby contributing to advances in healthcare.

A simple approach for quantizing neural networks

In this short note, we propose a new method for quantizing the weights of a fully trained neural network. A simple deterministic pre-processing step allows us to quantize network layers via memoryless scalar quantization while preserving the network performance on given training data. On one hand, the computational complexity of this pre-processing slightly exceeds that of state-of-the-art algorithms in the literature. On the other hand, our approach does not require any hyper-parameter tuning and, in contrast to previous methods, allows a plain analysis. We provide rigorous theoretical guarantees in the case of quantizing single network layers and show that the relative error decays with the number of parameters in the network if the training data behave well, e.g., if it is sampled from suitable random distributions. The developed method also readily allows the quantization of deep networks by consecutive application to single layers.

Quality of Security Guarantees for and with Physical Unclonable Functions and Biometric Secrecy Systems.

Unique digital circuit outputs, considered as physical unclonable function (PUF) circuit outputs, can facilitate a secure and reliable secret key agreement. To tackle noise and high correlations between the PUF circuit outputs, transform coding methods combined with scalar quantizers are typically applied to extract the uncorrelated bit sequences reliably. In this paper, we create realistic models for these transformed outputs by fitting truncated distributions to them. We also show that the state-of-the-art models are inadequate to guarantee a target reliability level for all PUF outputs, which also means that secrecy cannot be guaranteed. Therefore, we introduce a quality of security parameter to control the percentage of the PUF circuit outputs for which a target security level can be guaranteed. By applying the finite-length information theory results to a public ring oscillator output dataset, we illustrate that security guarantees can be provided for each bit extracted from any PUF device by eliminating only a small subset of PUF circuit outputs. Furthermore, we conversely show that it is not possible to provide reliability or security guarantees without eliminating any PUF circuit output. Our holistic methods and analyses can be applied to any PUF type, as well as any biometric secrecy system, with continuous-valued outputs to extract secret keys with low hardware complexity.

Subject Review: Various Techniques for Image Compression

Due to the quick development of multimedia technology, now can see the increasing amount of multimedia data that is being generated or transmitted over the internet. digital image is one form of multimedia data, that can take different formats and different sizes. And here comes the main objective of image compression in transmitting or storing the data in an effective manner by decreasing the redundant or irrelevant information without significant loss of visual quality in image data. Compression of the images is utilized in various services, including TV broadcasts, satellite imagery, martial communications, webinars, medical imaging, weather reporting, etc. This paper provides a survey of the main techniques in image compression, covering both lossy & lossless approaches. The techniques that depend on lossy compression lose some of the image details during compression, while lossless compression techniques keep the image information without losing. Vector and scalar Quantization, transform coding, block truncation Coding, etc. are all examples of lossy approaches. Run length coding, entropy encoding, statistical coding, etc. are all examples of lossless techniques. This paper will assist the researchers in learning about these techniques and choosing appropriate techniques for their work.

Enhanced variable universe fuzzy PID control of the active suspension based on expansion factor parameters adaption and genetic algorithm

The optimal value of the expansion factor and quantization factor parameters is the key to determine the performance of the variable domain fuzzy proportion-integral-derivative (PID) controller. In view of the problem that the relevant parameters are fixed, difficult to determine and cannot be adjusted adaptively in the traditional variable domain fuzzy PID control, a kind of enhanced variable domain fuzzy PID control (EVUFP) is proposed to improve the ride comfort of vehicles. First, based on the traditional variable domain fuzzy PID control, an adaptive expansion factor controller is constructed to determine and adjust adaptively the parameters of the expansion factor in real time. Then the genetic algorithm is used to optimize the quantization factor and scale factor parameters to seek the optimal value of relevant parameters. And the obtained optimal parameters are substituted into the Simulink model to realize the design of the EVUFP controller. The simulation experiment results under different working conditions show that the proposed EVUFP control strategy can reduce the root mean square (RMS) values of the body acceleration, suspension dynamic deflection and tire dynamic load by more than 54%, 50% and 23%, respectively, improve the ride comfort of vehicles, and provide a new idea for the development of active suspensions.

Similarity Search in the Blink of an Eye with Compressed Indices

Nowadays, data is represented by vectors. Retrieving those vectors, among millions and billions, that are similar to a given query is a ubiquitous problem, known as similarity search, of relevance for a wide range of applications. Graph-based indices are currently the best performing techniques for billion-scale similarity search. However, their random-access memory pattern presents challenges to realize their full potential. In this work, we present new techniques and systems for creating faster and smaller graph-based indices. To this end, we introduce a novel vector compression method, Locally-adaptive Vector Quantization (LVQ), that uses per-vector scaling and scalar quantization to improve search performance with fast similarity computations and a reduced effective bandwidth, while decreasing memory footprint and barely impacting accuracy. LVQ, when combined with a new high-performance computing system for graph-based similarity search, establishes the new state of the art in terms of performance and memory footprint. For billions of vectors, LVQ outcompetes the second-best alternatives: (1) in the low-memory regime, by up to 20.7x in throughput with up to a 3x memory footprint reduction, and (2) in the high-throughput regime by 5.8x with 1.4x less memory.

Learning Bilateral Clipping Parametric Activation for Low-Bit Neural Networks

Among various network compression methods, network quantization has developed rapidly due to its superior compression performance. However, trivial activation quantization schemes limit the compression performance of network quantization. Most conventional activation quantization methods directly utilize the rectified activation functions to quantize models, yet their unbounded outputs generally yield drastic accuracy degradation. To tackle this problem, we propose a comprehensive activation quantization technique namely Bilateral Clipping Parametric Rectified Linear Unit (BCPReLU) as a generalized version of all rectified activation functions, which limits the quantization range more flexibly during training. Specifically, trainable slopes and thresholds are introduced for both positive and negative inputs to find more flexible quantization scales. We theoretically demonstrate that BCPReLU has approximately the same expressive power as the corresponding unbounded version and establish its convergence in low-bit quantization networks. Extensive experiments on a variety of datasets and network architectures demonstrate the effectiveness of our trainable clipping activation function.

Busbar Voltage Control of DC Microgrid Group Based on Whale Optimization Algorithm and Fuzzy Control

Aiming at the problems of system response lag, poor anti-interference ability, and poor recovery ability of fuzzy dual closed-loop control, this paper proposes a fuzzy dual closed-loop control based on a whale optimization algorithm to optimize the control effect by simulating the foam attack mechanism of humpback whales, solving mathematical and structural optimization problems, adjusting quantization factor and scaling factor, and adjusting PI parameters in real-time. Matlab/Simulink is used to create the model. The experimental results show that the system has a quicker reaction, smaller overshoot, greater anti-interference capability, and improved robustness in various situations under the new control strategy.

A Secure and Effective Energy-Aware Fixed-Point Quantization Scheme for Asynchronous Federated Learning

Asynchronous federated learning (AsynFL) can effectively mitigate the impact of heterogeneity of edge nodes on joint training while satisfying participant user privacy protection and data security. However, the frequent exchange of massive data can lead to excess communication overhead between edge and central nodes regardless of whether the federated learning (FL) algorithm uses synchronous or asynchronous aggregation. Therefore, there is an urgent need for a method that can simultaneously take into account device heterogeneity and edge node energy consumption reduction. This paper proposes a novel Fixed-point Asynchronous Federated Learning (FixedAsynFL) algorithm, which could mitigate the resource consumption caused by frequent data communication while alleviating the effect of device heterogeneity. FixedAsynFL uses fixed-point quantization to compress the local and global models in AsynFL. In order to balance energy consumption and learning accuracy, this paper proposed a quantization scale selection mechanism. This paper examines the mathematical relationship between the quantization scale and energy consumption of the computation/communication process in the FixedAsynFL. Based on considering the upper bound of quantization noise, this paper optimizes the quantization scale by minimizing communication and computation consumption. This paper performs pertinent experiments on the MNIST dataset with several edge nodes of different computing efficiency. The results show that the FixedAsynFL algorithm with an 8-bit quantization can significantly reduce the communication data size by 81.3% and save the computation energy in the training phase by 74.9% without significant loss of accuracy. According to the experimental results, we can see that the proposed AsynFixedFL algorithm can effectively solve the problem of device heterogeneity and energy consumption limitation of edge nodes.

An Algorithm for Learning Orthonormal Matrix Codebooks for Adaptive Transform Coding.

This paper proposes a novel data-driven approach to designing orthonormal transform matrix codebooks for adaptive transform coding of any non-stationary vector processes which can be considered locally stationary. Our algorithm, which belongs to the class of block-coordinate descent algorithms, relies on simple probability models such as Gaussian or Laplacian for transform coefficients to directly minimize with respect to the orthonormal transform matrix the mean square error (MSE) of scalar quantization and entropy coding of transform coefficients. A difficulty commonly encountered in such minimization problems is imposing the orthonormality constraint on the matrix solution. We get around this difficulty by mapping the constrained problem in Euclidean space to an unconstrained problem on the Stiefel manifold and leveraging known algorithms for unconstrained optimization on manifolds. While the basic design algorithm directly applies to non-separable transforms, an extension to separable transforms is also proposed. We present experimental results for adaptive transform coding of still images and video inter-frame prediction residuals, comparing the transforms designed using the proposed method and a number of other content-adaptive transforms recently reported in the literature.

Design and Analysis of Hardware-Limited Non-Uniform Task-Based Quantizers

Hardware-limited task-based quantization is a new design paradigm for data acquisition systems equipped with serial scalar analog-to-digital converters using a small number of bits. By taking into account the underlying system task, task-based quantizers can efficiently recover the desired parameters from the low-bit quantized observation. Current design and analysis frameworks for hardware-limited task-based quantization are only applicable to inputs with bounded support and uniform quantizers with non-subtractive dithering. Here, we propose a new framework based on generalized Bussgang decomposition that enables the design and analysis of hardware-limited task-based quantizers that are equipped with non-uniform scalar quantizers or that have inputs with unbounded support. We first consider the scenario in which the task is linear. Under this scenario, we derive new pre-quantization and post-quantization linear mappings for task-based quantizers with mean squared error (MSE) that closely matches the theoretical MSE. Next, we extend the proposed analysis framework to quadratic tasks. We demonstrate that our derived analytical expression for the MSE accurately predicts the performance of task-based quantizers with quadratic tasks.