Scalar Quantization Research Articles

Iteratively consistent one-bit phase retrieval

We introduce the ICQPhase algorithm for iteratively consistent quantized phase retrieval. ICQPhase addresses the problem of recovering a signal [Formula: see text] or [Formula: see text] from one-bit quantized phaseless measurements [Formula: see text], [Formula: see text], where [Formula: see text] are orthogonal projections and where [Formula: see text] is a one-bit scalar quantizer. We numerically validate the performance of ICQPhase and demonstrate that it experimentally achieves mean squared error of order [Formula: see text] in a variety of settings. We provide a theoretical analysis of the algorithm for the case of rank-one projections of signals in two-dimensional real space and prove that the mean squared error is of order [Formula: see text].

Performance Analysis of a 2-bit Dual-Mode Uniform Scalar Quantizer for Laplacian Source

The main issue when dealing with the non-adaptive scalar quantizers is their sensitivity to variance-mismatch, the effect that occurs when the data variance differs from the one used for the quantizer design. In this paper, we consider the influence of that effect in low-rate (2-bit) uniform scalar quantization (USQ) of Laplacian source and also we propose adequate measure to suppress it. Particularly, the approach we propose represents the upgraded version of the previous approaches used to improve performance of the single quantizer. It is based on dual-mode quantization that combines two 2-bit USQs (with adequately chosen parameters) to process input data, selected by applying the special rule. Analysis conducted in theoretical domain has shown that the proposed approach is less sensitive to variance-mismatch, making the dual-mode USQ more efficient in terms of robustness than the single USQ. Also, a gain is achieved compared to other 2-bit quantizer solutions. Experimental results are also provided for quantization of weights of the multi-layer perceptron (MLP) neural network, where good matching with the theoretical results is observed. Due to these achievements, we believe that the solution we propose can be a good choice for compression of non-stationary data modeled by Laplacian distribution, such as neural network parameters.

Two Interval Upper-Bound Q-Function Approximations with Applications

The Gaussian Q-function has considerable applications in numerous areas of science and engineering. However, the fact that a closed-form expression for this function does not exist encourages finding approximations or bounds of the Q-function. In this paper, we determine analytically two novel interval upper bound Q-function approximations and show that they could be used efficiently not only for the symbol error probability (SEP) estimation of transmission over Nakagami-m fading channels, but also for the average symbol error probability (ASEP) evaluation for two modulation formats. Specifically, we determine analytically the composition of the upper bound Q-function approximations specified at disjoint intervals of the input argument values so as to provide the highest accuracy within the intervals, by utilizing the selected one of two upper bound Q-function approximations. We show that a further increase of the accuracy, achieved in the case with two upper-bound approximations composing the interval approximation, can be obtained by forming a composite interval approximation of the Q-function that assumes another extra interval and by specifying the third form for the upper-bound Q-function approximation. The proposed analytical approach can be considered universal and widely applicable. The results presented in the paper indicate that the proposed Q-function approximations outperform in terms of accuracy other well-known approximations carefully chosen for comparison purposes. This approximation can be used in numerous theoretical communication problems based on the Q-function calculation. In this paper, we apply it to estimate the average bit error rate (ABER), when the transmission in a Nakagami-m fading channel is observed for the assumed binary phase-shift keying (BPSK) and differentially encoded quadrature phase-shift keying (DE-QPSK) modulation formats, as well as to design scalar quantization with equiprobable cells for variables from a Gaussian source.

Photonics-Assisted Millimeter-Wave Communication System Based on Low-Bit Gaussian Mixture Model Adaptive Vector Quantization

The quantization technique with a low-bit resolution can significantly reduce the cost and power consumption of analog-to-digital converter (ADC). It will play an important role in energy conservation and cost reduction for the incoming B5G millimeter-wave (MMW) communication systems. In this paper, we propose and demonstrated experimentally a low-bit Gaussian mixture model (GMM) based non-uniform adaptive vector quantization (AVQ) scheme for the low-cost intensity modulated envelope detection photonics-assisted 28 GHz MMW communication system for the first time. The principles of GMM-based one-dimensional adaptive scalar quantization (ASQ) and multi-dimensional AVQ are first introduced, and then are used to realize the low-bit non-uniform adaptive quantization for reducing the ADC bit resolution of MMW receiver. Furthermore, the performance of traditional uniform quantization, the present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means and proposed GMM-based non-uniform ASQ/AVQ schemes are evaluated and compared in detail. Utilizing the proposed GMM-based AVQ scheme, the ADC quantization resolution in our MMW receiver can be reduced from 5 bits of the traditional uniform quantization to as low as 2 bits, without noticeable performance penalty. Moreover, as compared with the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means-based quantization scheme, the MMW receiver enabled by GMM-based ASQ/AVQ scheme can save about half of the quantization time under similar performance. This is mainly because the clustering based on probability converges faster than the Euclidean distance, which significantly reduces the number of iterations required. Therefore, the GMM-based AVQ scheme is a promising solution to realize high performance ADCs with low-bit resolution for future MMW-enabled optical fiber wireless access networks.

Data reduction through optimized scalar quantization for more compact neural networks

Raw data generation for several existing and planned large physics experiments now exceeds TB/s rates, generating untenable data sets in very little time. Those data often demonstrate high dimensionality while containing limited information. Meanwhile, Machine Learning algorithms are now becoming an essential part of data processing and data analysis. Those algorithms can be used offline for post processing and post data analysis, or they can be used online for real time processing providing ultra low latency experiment monitoring. Both use cases would benefit from data throughput reduction while preserving relevant information: one by reducing the offline storage requirements by several orders of magnitude and the other by allowing ultra fast online inferencing with low complexity Machine Learning models. Moreover, reducing the data source throughput also reduces material cost, power and data management requirements. In this work we demonstrate optimized nonuniform scalar quantization for data source reduction. This data reduction allows lower dimensional representations while preserving the relevant information of the data, thus enabling high accuracy Tiny Machine Learning classifier models for online fast inferences. We demonstrate this approach with an initial proof of concept targeting the CookieBox, an array of electron spectrometers used for angular streaking, that was developed for LCLS-II as an online beam diagnostic tool. We used the Lloyd-Max algorithm with the CookieBox dataset to design an optimized nonuniform scalar quantizer. Optimized quantization lets us reduce input data volume by 69% with no significant impact on inference accuracy. When we tolerate a 2% loss on inference accuracy, we achieved 81% of input data reduction. Finally, the change from a 7-bit to a 3-bit input data quantization reduces our neural network size by 38%.

Analysis of Compressing PAPR-Reduced OFDM IQ Samples for Cloud Radio Access Network

A common problem of the virtualized cloud radio access network architecture (C-RAN) is the compression of the time-domain IQ samples before transmission over the fronthaul link. Considering a multicarrier waveform such as OFDM, whose IQ samples follow a quasi-Gaussian distribution, the conventional Gaussian quantizer may be used as the optimal solution to the compression problem. However, since the high peak-to-average power ratio (PAPR) of OFDM signals remains a serious problem, various techniques may be employed to reduce the time-domain fluctuations of the IQ samples in the OFDM, resulting in a change in its distribution. The latter fact makes the Gaussian quantizer suboptimal. The literature lacks a performance analysis of the conventional OFDM-based compression techniques when the PAPR of the OFDM signal is reduced. Therefore, in this paper, we study for the first time the impact of reducing the PAPR of the OFDM signal before compression in the C-RAN architecture through rate-distortion analysis. We consider clipping and tone reservation PAPR reduction algorithms. The former is the simplest PAPR reduction approach, while the latter is one of the most effective algorithms used in broadcasting standards such as DVB-T2 and ATSC 3.0. We first derive the distribution of the PAPR-reduced OFDM IQ samples. This is used to optimize the thresholds and codebook levels of a non-uniform scalar quantizer and the number of quantization bits allocated for each quantized level in the entropy coding stage, along with the MER performance analysis. The simulation results show that the conventional Gaussian-based compression techniques applied to a PAPR-reduced signal is not very robust to the statistical changes in the signal unless the signal distribution at the input of the Gaussian quantizer is not significantly affected. However, a significant gain is obtained when the quantizer is optimized with respect to the true distribution of the PAPR-reduced IQ samples.

EPQuant: A Graph Neural Network compression approach based on product quantization

Graph Neural Networks (GNNs) have been widely used in graph analysis due to their strong performance on a wide variety of tasks. Unfortunately, as the size of graphs keeps growing, large graphs can easily consume Terabytes, and training on such graphs may take days. The high memory footprint limits the usage of GNNs on resource-constrained devices like smartphones and IoT devices. Hence, reducing the storage cost, training time, and inference latency is highly desirable. In this work, we apply Product Quantization (PQ) on GNNs for the first time to achieve superior memory capacity reduction. To alleviate the processing burden caused by PQ and improve compression performance, we propose Enhanced Product Quantization (EPQ). It reduces the input graph data, which tends to dominate memory consumption, and accelerates clustering in PQ. Moreover, an efficient quantization framework11Github repository:https://github.com/Lyun-Huang/EPQuant for GNNs is proposed, which combines EPQ with Scalar Quantization (SQ), to achieve improved compression performance and computation acceleration on off-the-shelf hardware, enabling the deployment of GNNs on resource-constrained devices. In addition, the proposed quantization framework can be applied to the existing GNNs without too much porting effort. Extensive experimental results show that the proposed quantization scheme can achieve 321.26× and 184.03× memory capacity compression for input graph data and overall storage, respectively. This impressive memory reduction can come with an accuracy loss of less than 1%.

Closed-Form Sum-Rate Analysis of Interference Alignment with Limited Feedback Based on Scalar Quantization and Random Vector Quantization

Interference alignment (IA) is a promising interference management technique to achieve the theoretical optimal degree of freedom (DoF) performance in multi-user cooperation scenarios. However, the effective achievable sum-rate performance of IA is largely affected by the feedback overhead and accuracy of channel state information (CSI) and decoding information (DI). Therefore, it is critical to establish the exact relationship between feedback overhead and the achievable sum-rate of IA to obtain the optimal effective performance. Most existing IA performance analysis approaches focus on the vector quantization (VQ)-based feedback strategy, but the implementation complexity of VQ will be excessive when more quantization bits are required to achieve the expected quantization accuracy for larger-sized matrices or higher signal-to-noise ratio (SNR) regimes. Moreover, the obtained achievable sum-rate formulas are too complicated for quick performance evaluation. In this paper, a new sum-rate performance analysis method for IA under different quantization and feedback strategies is proposed to achieve a trade-off between accuracy and complexity, and the closed-form achievable sum-rate expressions are derived. First, in the IA case with random vector quantization (RVQ)-based CSI feedback, the quantization error of RVQ is transformed into the equivalent VQ error of the Gaussian channel error, based on which the achievable sum-rate formula is obtained. Second, in the IA case with scalar quantization (SQ)-based CSI feedback, the relationship between the effective sum-rate and SQ bits is established. Third, in the IA case with SQ-based CSI feedback and RVQ-based DI feedback, the achievable sum-rate formula is derived by combining these two kinds of quantization errors. Finally, the simulation results confirm that the theoretical results are accurate enough, which can help to determine the optimal CSI feedback overhead in practical channel conditions. Moreover, the theoretical and simulation results demonstrate that RVQ may be more applicable to IA scenarios with fewer receiving antennas and low SNR regimes.

Mastering Quantization is Key for Video Compression

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">This article discusses some key elements and solutions for efficient video compression. More precisely, we introduce key differentiating techniques for video signal quantization within modern hybrid video codecs. First, we recall the fundamentals of scalar quantization and rate-distortion (R-D) theory and then describe the different control points and levels of granularity for optimizing signal quantization in modern video compression standards. From this common knowledge, we discuss approaches in how to jointly optimize those various levels of quantization refinements for improved video encoding quality. Most notably, we explain how to take advantage of a look-ahead module (available in most industrial encoder implementations) to model spatio-temporal coding dependencies, and how to further compute optimal quantization information from an R-D standpoint for a group of pictures (GOPs). Complementary to this first approach, we share some insights about a local quantization refinement (LQR) algorithm. Such an algorithm is often ignored in practical encoder implementation due to its apparent complexity; we develop how and why it can efficiently work in realtime software encoding</i> .

An Adaptive Rate Allocation Scheme for Time-Varying Graph Signal Quantization

To address the communication resource limitations the uplink data in some distributed networks suffers from, quantization enables these graph signals to realize compression. However, the compression process is accompanied by quantization errors, which pose threat to the communication quality. In addition, in real-world scenarios, graph signals tend to evolve with time, where the information loss would be larger without adaption to the evolvement. To tackle the above problems, we first propose an adaptive rate allocation scheme, which allocates rate to each quantizer under a total rate constraint, for time-varying graph signals. Along with the smoothness of graph signals at the same time instant and the rate-distortion features of scalar quantization, the smoothly evolving characteristics of time-varying graph signals are leveraged to adaptively adjust the allocated rate with time to reduce quantization distortion of uplink data incurred by the time-varying feature. Simulation results demonstrate the superiority of distortion performance of the proposed rate allocation scheme on both synthetic and real-world graphs.

Quantization for Spectral Super-Resolution

We show that the method of distributed noise-shaping beta-quantization offers superior performance for the problem of spectral super-resolution with quantization whenever there is redundancy in the number of measurements. More precisely, we define the over-sampling ratio \(\lambda \) as the largest integer such that \(\lfloor M/\lambda \rfloor - 1\ge 4/\Delta \), where M denotes the number of Fourier measurements and \(\Delta \) is the minimum separation distance associated with the atomic measure to be resolved. We prove that for any number \(K\ge 2\) of quantization levels available for the real and imaginary parts of the measurements, our quantization method combined with either TV-min/BLASSO or ESPRIT guarantees reconstruction accuracy of order \(O(M^{1/4}\lambda ^{5/4} K^{- \lambda /2})\) and \(O(M^{3/2} \lambda ^{1/2} K^{- \lambda })\), respectively, where the implicit constants are independent of M, K and \(\lambda \). In contrast, naive rounding or memoryless scalar quantization for the same alphabet offers a guarantee of order \(O(M^{-1}K^{-1})\) only, regardless of the reconstruction algorithm.

A Rate Control Model of MPEG-4 Encoder for Video Transmission over Wireless Sensor Network

Recently, multimedia application has a lot of attention in the research community, especially when transmitting video over IEEE 802.15.4 standard. This is due to the capability of providing low complexity with low cost, but still maintaining the quality of video in term of packet received. However, transmitting video over Wireless Sensor Network (WSN) posed a new research challenges with high bandwidth demand and energy constrained of sensor nodes. MPEG-4 video codec is one of the compression techniques that used to decrease the amount of bandwidth required to meet WSN environment. Therefore, video encoding is a useful tool for rate control to control the video bit rate and maintaining the video quality especially in real-time communication applications. Video bit rate is affected by quantization scale, frame rate, and Group of Picture (GOP) size. A rate control model called enhanced Video Motion Classification based (e-ViMoC) model is proposed in this paper to produce the desired bit rate that complies to the IEEE 802.15.4 standard, while at the same time preserving the video quality. The analysis has shown that, the video transmission using e-ViMoC rate control achieves enhancement in delivery ratio, energy consumption and video quality (PSNR) when compared to video transmission using uncompressed video.

Speaker Recognition Using Constrained Convolutional Neural Networks in Emotional Speech

Speaker recognition is an important classification task, which can be solved using several approaches. Although building a speaker recognition model on a closed set of speakers under neutral speaking conditions is a well-researched task and there are solutions that provide excellent performance, the classification accuracy of developed models significantly decreases when applying them to emotional speech or in the presence of interference. Furthermore, deep models may require a large number of parameters, so constrained solutions are desirable in order to implement them on edge devices in the Internet of Things systems for real-time detection. The aim of this paper is to propose a simple and constrained convolutional neural network for speaker recognition tasks and to examine its robustness for recognition in emotional speech conditions. We examine three quantization methods for developing a constrained network: floating-point eight format, ternary scalar quantization, and binary scalar quantization. The results are demonstrated on the recently recorded SEAC dataset.

Research on Temperature Control of Fuel-Cell Cooling System Based on Variable Domain Fuzzy PID

To ensure the energy conversion efficiency of a proton-exchange membrane fuel cell (PEMFC), it is necessary to establish a water-cooled cooling system to keep the inlet temperature of fuel-cell coolant and the temperature difference between the inlet and outlet temperature within the set range. First, a semi-empirical and semi-mechanism model was built in Simulink. Then, a variable-universe fuzzy PID controller was designed to adjust the quantization factor and scaling factor by scaling factor α1, α2 and β to improve the accuracy of the control results. Finally, the cooling system model with controller is simulated. The results show that compared with other control methods, the simulation results of the variable-universe fuzzy PID controller have smaller maximum overshoot, faster response speed and more gentle temperature fluctuation.

Analysis of the Measurement Matrix in Directional Predictive Coding for Compressive Sensing of Medical Images

Compressive sensing of 2D signals involves three fundamental steps: sparse representation, linear measurement matrix, and recovery of the signal. This paper focuses on analyzing the efficiency of various measurement matrices for compressive sensing of medical images based on theoretical predictive coding. During encoding, the prediction is efficiently chosen by four directional predictive modes for block-based compressive sensing measurements. In this work, Gaussian, Bernoulli, Laplace, Logistic, and Cauchy random matrices are used as the measurement matrices. While decoding, the same optimal prediction is de-quantized. Peak-signal-to-noise ratio and sparsity are used for evaluating the performance of measurement matrices. The experimental result shows that the spatially directional predictive coding (SDPC) with Laplace measurement matrices performs better compared to scalar quantization (SQ) and differential pulse code modulation (DPCM) methods. The results indicate that the Laplace measurement matrix is the most suitable in compressive sensing of medical images.

Nonparametric Decentralized Detection and Sparse Sensor Selection via Multi-Sensor Online Kernel Scalar Quantization

Signal classification problems arise in a wide variety of applications, and their demand is only expected to grow. In this paper, we focus on the wireless sensor network signal classification setting, where each sensor forwards quantized signals to a fusion center to be classified. Our primary goal is to train a decision function and quantizers across the sensors to maximize the classification performance in an online manner. Moreover, we are interested in sparse sensor selection using a marginalized weighted kernel approach to improve network resource efficiency by disabling less reliable sensors with minimal effect on classification performance.To achieve our goals, we develop a multi-sensor online kernel scalar quantization (MSOKSQ) learning strategy that operates on the sensor outputs at the fusion center. Our theoretical analysis reveals how the proposed algorithm affects the quantizers across the sensors. Additionally, we provide a convergence analysis of our online learning approach by studying its relationship to batch learning. We conduct numerical studies under different classification and sensor network settings which demonstrate the accuracy gains from optimizing different components of MSOKSQ and robustness to reduction in the number of sensors selected.

Research on Optimization of Fuzzy Network Control System Based on New Smith Predictive Time Delay Compensation

Aiming at the problem of network delay when the network control system transmits data, this paper adopts a new type of fuzzy control method of Smith predictor to compensate for the delay. In actual application scenarios, it is difficult to accurately match the Smith prediction model with the actual model. At the same time, the quantization factor and scale factor in the fuzzy PID controller are too dependent on experience, which makes the system’s adaptability to actual working conditions very poor. In this paper, genetic algorithm is used to optimizes fuzzy PID. According to the simulation results, when the Smith predictive model does not match the actual model, the steady-state performance and dynamic performance of the system under the fuzzy PID control optimized by the genetic algorithm are improved.

Art Image Compression Based on Lossless LZW Hashing Ciphering Algorithm

Color image compression is a good way to encode digital images by decreasing the number of bits wanted to supply the image. The main objective is to reduce storage space, reduce transportation costs and maintain good quality. In current research work, a simple effective methodology is proposed for the purpose of compressing color art digital images and obtaining a low bit rate by compressing the matrix resulting from the scalar quantization process (reducing the number of bits from 24 to 8 bits) using displacement coding and then compressing the remainder using the Mabel ZF algorithm Welch LZW. The proposed methodology maintains the quality of the reconstructed image. Macroscopic and quantitative experimental results on technical color images show that the proposed methodology gives reconstructed images with a high PSNR value compared to standard image compression techniques.

Integer-Only CNNs with 4 Bit Weights and Bit-Shift Quantization Scales at Full-Precision Accuracy

Quantization of neural networks has been one of the most popular techniques to compress models for embedded (IoT) hardware platforms with highly constrained latency, storage, memory-bandwidth, and energy specifications. Limiting the number of bits per weight and activation has been the main focus in the literature. To avoid major degradation of accuracy, common quantization methods introduce additional scale factors to adapt the quantized values to the diverse data ranges, present in full-precision (floating-point) neural networks. These scales are usually kept in high precision, requiring the target compute engine to support a few high-precision multiplications, which is not desirable due to the larger hardware cost. Little effort has yet been invested in trying to avoid high-precision multipliers altogether, especially in combination with 4 bit weights. This work proposes a new quantization scheme, based on power-of-two quantization scales, that works on-par compared to uniform per-channel quantization with full-precision 32 bit quantization scales when using only 4 bit weights. This is done through the addition of a low-precision lookup-table that translates stored 4 bit weights into nonuniformly distributed 8 bit weights for internal computation. All our quantized ImageNet CNNs achieved or even exceeded the Top-1 accuracy of their full-precision counterparts, with ResNet18 exceeding its full-precision model by 0.35%. Our MobileNetV2 model achieved state-of-the-art performance with only a slight drop in accuracy of 0.51%.

Texture Feature Extraction of Lumbar Spine Trabecular Bone Radiograph Image using Laplacian of Gaussian Filter with KNN Classification to Diagnose Osteoporosis

The human bones are categorized based on elemental micro architecture and porosity. The porosity of the inner trabecular bone is high that is 40-95% and the nature of the bone is soft and spongy whereas the cortical bone is harder and is less porous that is 5 to 15%. Osteoporosis is a disease that normally affects women usually after their menopause. It largely causes mild bone fractures and further stages lead to the demise of an individual. The detection of Osteoporosis in Lumbar Spine has been widely recognized as a promising way to frequent fractures. Therefore, premature analysis of osteoporosis will estimate the risk of the bone fracture which prevents life threats. The paper is systematized in two different sections to classify normal (non-osteoporosis) and abnormal(osteoporosis)Lumbar spine trabecular bone. In this method, the first section is based on discriminating the lumbar spine trabecular bone micro-architecture predisposing by means of first and second order directional derivative of Laplacian of Gaussian filter with different standard deviation to acquire the minimum and maximum responses. The dimension reduction of texture features, quantization and adjacent scale coding with weighted multipliers are used to lessen the intensity variations of texture features. The second section is based on the reduction of histogram features as a training data set for classification of normal and osteoporotic images of lumbar spine (L1-L4) using K-Nearest Neighborhood (KNN) classifier. The tested dataset result gives effective classification accuracy of 97.22% with lesser texture feature dimension. The usage of weight multiplier as well as quantization technique plays a major role for the improvement of accuracy to diagnose osteoporosis for an input noisy and noiseless image.