Efficient Quantization Research Articles

SnowPac: a multiscale cubic B-spline wavelet compressor for astronomical images

ABSTRACT As more advanced and complex survey telescopes are developed, the size and scale of data being captured grows at increasing rates. Across various domains, data compression through wavelets has enabled the reduction of data size and increase in computation efficiency. In this paper, we provide qualitative and quantitative tests of a new wavelet-based image compression method compared against the current standard for astronomical images. The analysis is improved by making use of state-of-the-art object detection systems to accurately measure the impact of the compression. We find that a combination of lossy wavelet-based methods, efficient quantization, and lossless dictionary compressors can preserve up to 98 per cent of astronomical objects at a 10:1 compression ratio. This significant reduction in file size also preserves astronomical object properties better than existing methods. These methods help further reduce future workloads for image-heavy processing pipelines.

Image‐independent optimal non‐negative integer bit allocation technique for the DCT‐based image transform coders

The optimum non-negative integer bit allocation (ONIBA) is an important technique, which provides optimal quantisation of transform coefficients for the image transform coders (ITCs). However, the existing ONIBA algorithms are still not popular for the discrete cosine transform (DCT)-based ITCs, due to their image-dependent nature and additional side information requirements. Therefore, this study presents a novel image-independent ONIBA (IIONIBA) technique to achieve efficient quantisation for the DCT-ITCs. For the development of the proposed IIONIBA technique, initially, an image-dependent ONIBA algorithm is proposed, which is then mapped into desired image-independent solution via utilisation of a prepared combined image and proposed modified step size mapping technique. Thereafter, a new lookup table for the elements of quantisation tables, obtained from the proposed IIONIBA technique, is established using non-linear regression analysis, to reduce the problem of additional side information requirements. Several experiments are performed to evaluate the performance of the proposed IIONIBA technique based on the visual quality assessment of reconstructed images and the image quality indexes peak signal to noise ratio (PSNR) and mean structural similarity index (MSSIM). The results show that the proposed IIONIBA technique delivers better quantisation and provides significant gains in the image quality indexes as compared to the recent quantisation techniques.

Efficient Quantizer Design Algorithm for Sequence-Based Localization (SBL) Systems

Efficient Quantizer Design Algorithm for Sequence-Based Localization (SBL) Systems

A Low-Cost VLSI Architecture of the Bilateral Filter for Real-Time Image Denoising

In this paper, a low-cost hardware architecture of the bilateral filter for real-time image processing is proposed. Based on the techniques of distance-oriented grouping and hardware resource sharing, the usage of multipliers can decrease 48% as compared to the previous approach. Besides, an efficient quantization method is applied to reduce the size of required look up tables. The experimental results show that the proposed architecture is cost efficient while maintaining the same image quality, frame rate and working clock frequency.

Probability Density Rank-Based Quantization for Convex Universal Learning Machines.

The distributions of input data are very important for learning machines, such as the convex universal learning machines (CULMs). The CULMs are a family of universal learning machines with convex optimization. However, the computational complexity is a crucial problem in CULMs, because the dimension of the nonlinear mapping layer (the hidden layer) of the CULMs is usually rather large in complex system modeling. In this article, we propose an efficient quantization method called Probability density Rank-based Quantization (PRQ) to decrease the computational complexity of CULMs. The PRQ ranks the data according to the estimated probability densities and then selects a subset whose elements are equally spaced in the ranked data sequence. We apply the PRQ to kernel ridge regression (KRR) and random Fourier feature recursive least squares (RFF-RLS), which are two typical algorithms of CULMs. The proposed method not only keeps the similarity of data distribution between the code book and data set but also reduces the computational cost by using the kd-tree. Meanwhile, for a given data set, the method yields deterministic quantization results, and it can also exclude the outliers and avoid too many borders in the code book. This brings great convenience to practical applications of the CULMs. The proposed PRQ is evaluated on several real-world benchmark data sets. Experimental results show satisfactory performance of PRQ compared with some state-of-the-art methods.

Efficient Quantization for Neural Networks with Binary Weights and Low Bitwidth Activations

Quantization has shown stunning efficiency on deep neural network, especially for portable devices with limited resources. Most existing works uncritically extend weight quantization methods to activations. However, we take the view that best performance can be obtained by applying different quantization methods to weights and activations respectively. In this paper, we design a new activation function dubbed CReLU from the quantization perspective and further complement this design with appropriate initialization method and training procedure. Moreover, we develop a specific quantization strategy in which we formulate the forward and backward approximation of weights with binary values and quantize the activations to low bitwdth using linear or logarithmic quantizer. We show, for the first time, our final quantized model with binary weights and ultra low bitwidth activations outperforms the previous best models by large margins on ImageNet as well as achieving nearly a 10.85× theoretical speedup with ResNet-18. Furthermore, ablation experiments and theoretical analysis demonstrate the effectiveness and robustness of CReLU in comparison with other activation functions.

Improved Optimum Nonnegative Integer Bit Allocation Algorithm Using Fuzzy Domain Variance Estimation and Refinement for the Wavelet-Based Image Compression

Optimum nonnegative integer bit allocation (ONIBA) is a conspicuous technique, which usually provides the solution of optimal quantization issues for the transform coders (TCs). In order to obtain the optimum bits for a specific quantizer, all the existing ONIBA algorithms strongly rely on the variance characteristics of transform coefficients. Typically, in the wavelet-based TCs, the sub-band variances are directly estimated in the wavelet domain. This direct variance estimation is not supposed to be the best way to obtain the exact variance information, because the practical values of the wavelet coefficients may not be precise and therefore constitute an uncertain environment for the accurate variance estimation. Consequently, all the existing ONIBA algorithms often exhibit poor quantization performance in the presence of entropy coder. Hence, this paper presents a new fuzzy domain variance estimation and refinement (FDVER)-based ONIBA algorithm to attain the real optimum quantization of the wavelet coefficients in the presence of entropy coder. The outcome shows that the proposed FDVER-ONIBA algorithm outperforms and provides high-quality image compression along with the significant bitrate savings by the efficient quantization of the wavelet coefficients as compared to the existing common sub-band coding technique and the recent ONIBA algorithms.

Space Efficient Quantization for Deep Convolutional Neural Networks

Deep convolutional neural networks (DCNNs) have shown outstanding performance in the fields of computer vision, natural language processing, and complex system analysis. With the improvement of performance with deeper layers, DCNNs incur higher computational complexity and larger storage requirement, making it extremely difficult to deploy DCNNs on resource-limited embedded systems (such as mobile devices or Internet of Things devices). Network quantization efficiently reduces storage space required by DCNNs. However, the performance of DCNNs often drops rapidly as the quantization bit reduces. In this article, we propose a space efficient quantization scheme which uses eight or less bits to represent the original 32-bit weights. We adopt singular value decomposition (SVD) method to decrease the parameter size of fully-connected layers for further compression. Additionally, we propose a weight clipping method based on dynamic boundary to improve the performance when using lower precision. Experimental results demonstrate that our approach can achieve up to approximately 14x compression while preserving almost the same accuracy compared with the full-precision models. The proposed weight clipping method can also significantly improve the performance of DCNNs when lower precision is required.

A Hardware Trojan Detection Method Based on Structural Features of Trojan and Host Circuits

Modern IC designs often involve outsourced IP cores. It is convinced that there are opportunities in which the IP cores contain malicious logic, namely hardware Trojan (HT), which raises serious concerns about the trustworthiness of ICs used in mission-critical applications. This paper proposes an HT detection method by analyzing the combined structural features of HTs and host circuits. The structural features of combinational and sequential logic HTs are extracted and form an HT feature database. An efficient quantization approach on feature matching is proposed to search the features from circuit designs. The experiments conducted on TrustHub benchmarks show that the proposed method can successfully detect all stealth Trojans with runtime within 72 s on a platform with an AMD 2.00-GHz CPU with 4-GB RAM and a low false positive rate compared with the existing methods.

Quantized Minimum Error Entropy Criterion.

Comparing with traditional learning criteria, such as mean square error, the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyi's entropy estimator, called information potential (IP), is a popular MEE cost in information theoretic learning. The computational complexity of IP is, however, quadratic in terms of sample number due to double summation. This creates the computational bottlenecks, especially for large-scale data sets. To address this problem, in this paper, we propose an efficient quantization approach to reduce the computational burden of IP, which decreases the complexity from O(N2) to O(MN) with M << N . The new learning criterion is called the quantized MEE (QMEE). Some basic properties of QMEE are presented. Illustrative examples with linear-in-parameter models are provided to verify the excellent performance of QMEE.

Green fronthaul allocation and power management in Cloud-RAN

In cloud radio access network (Cloud-RAN), most previous research aims to reduce the energy consumption of the total system and ignores specific requirements of energy-saving in mobile users. For the meantime, some efforts have been made to devise efficient signal quantization techniques for fronthaul capacity restriction to improve the performance of Cloud-RAN. To improve the user-centric energy efficiency (EE), we propose two kinds of optimization algorithms in orthogonal frequency division multiple access-based Cloud-RAN in this paper, whereby Gaussian quantization and uniform scalar quantization methods are utilized. For single-RRH scenario, alternative iterative optimization algorithms are proposed to maximize users’ EE, by jointly allocating resources. Furthermore, in more general scenario for multiple RRHs, authors investigate a modified particle swarm optimization (M-PSO) algorithm to solve more complex problems, which remain non-linear and non-convex. Simulation results demonstrate the proposed schemes in this paper outperform conventional solutions and improve EE of users.

Efficient Quantization with Linear Index Coding for Deep‐Space Images

Due to inevitable propagation delay involved in deep‐space communication systems, very high cost is associated with the retransmission of erroneous segments. Quantization with linear index coding (QLIC) scheme is known to provide compression along with robust transmission of deep‐space images, and thus the likelihood of retransmissions is significantly reduced. This paper aims to improve its spectral efficiency as well as robustness. First, multiple quantization refinement levels per transmitted source block of QLIC are proposed to increase spectral efficiency. Then, iterative multipass decoding is introduced to jointly decode the subsource symbol‐planes. It achieves better PSNR of the reconstructed image as compared to the baseline one‐pass decoding approach of QLIC.

Efficient quantization and fixed-point representation for MIMO turbo-detection and turbo-demapping

In the domain of wireless digital communication, floating-point arithmetic is generally used to conduct performance evaluation studies of algorithms. This is typically limited to theoretical performance evaluation in terms of communication quality and error rates. For a practical implementation perspective, using fixed-point arithmetic instead of floating-point reduces significantly implementation costs in terms of area occupation and energy consumption. However, this implies a complex conversion process, particularly if the considered algorithm includes complex arithmetic operations with high accuracy requirements and if the target system presents many configuration parameters. In this context, the purpose of the paper is to present an efficient quantization and fixed-point representation for turbo-detection and turbo-demapping. The impact of floating-to-fixed-point conversion is illustrated upon the error-rate performance of the receiver for different system configurations. Only a slight degradation in the error-rate performance of the receiver is observed when implementing the detector and demapper modules which utilize the devised quantization and fixed-point arithmetic rather than floating-point arithmetic.

Selective bit embedding scheme for robust blind color image watermarking

In this paper, we propose a novel robust blind color image watermarking method, namely SMLE, that allows to embed a gray-scale image as watermark into a host color image in the wavelet domain. After decomposing the gray-scale watermark to component binary images in digits ordering from least significant bit (LSB) to most significant bit (MSB), the retrieved binary bits are then embedded into wavelet blocks of two optimal color channels by using an efficient quantization technique, where the wavelet coefficient difference in each block is quantized to either two pre-defined thresholds for corresponding 0-bits and 1-bits. To optimize the watermark imperceptibility, we equally split the coefficient modified quantity on two middle-frequency sub-bands instead of only one as in existing approaches. The improvement of embedding rule increases approximately 3 dB of watermarked image quality. An adequate trade-off between robustness and imperceptibility is controlled by a factor representing the embedding strength. As for extraction process, we exploit 2D Otsu algorithm for higher accuracy of watermark detection than that of 1D Otsu. Experimental results prove the robustness of our SMLE watermarking model against common image processing operations along with its efficient retention of the imperceptibility of the watermark in the host image. Compared to state-of-the-art methods, our approach outperforms in most of robustness tests at a same high payload capacity.

Accelerating Recurrent Neural Networks: A Memory-Efficient Approach

Recurrent neural networks (RNNs) have achieved the state-of-the-art performance on various sequence learning tasks due to their powerful sequence modeling capability. However, RNNs usually require a large number of parameters and high computational complexity. Hence, it is quite challenging to implement complex RNNs on embedded devices with stringent memory and latency requirement. In this paper, we first present a novel hybrid compression method for a widely used RNN variant, long-short term memory (LSTM), to tackle these implementation challenges. By properly using circulant matrices, forward nonlinear function approximation, and efficient quantization schemes with a retrain-based training strategy, the proposed compression method can reduce more than 95% of memory usage with negligible accuracy loss when verified under language modeling and speech recognition tasks. An efficient scalable parallel hardware architecture is then proposed for the compressed LSTM. With an innovative chessboard division method for matrix-vector multiplications, the parallelism of the proposed hardware architecture can be freely chosen under certain latency requirement. Specifically, for the circulant matrix-vector multiplications employed in the compressed LSTM, the circulant matrices are judiciously reorganized to fit in with the chessboard division and minimize the number of memory accesses required for the matrix multiplications. The proposed architecture is modeled using register transfer language (RTL) and synthesized under the TSMC 90-nm CMOS technology. With 518.5-kB on-chip memory, we are able to process a 512×512 compressed LSTM in 1.71 μs, corresponding to 2.46 TOPS on the uncompressed one, at a cost of 30.77-mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> chip area. The implementation results demonstrate that the proposed design can achieve significantly high flexibility and area efficiency, which satisfies many real-time applications on embedded devices. It is worth mentioning that the memory-efficient approach of accelerating LSTM developed in this paper is also applicable to other RNN variants.

Quantization and security enabled compressive video CODEC for WSN

Real time image and video transmission in surveillance applications need efficient video coding and security. Recent advances in computer vision have enabled the development of highly secured compressive video codec framework based on compressed sensing (CS) for video surveillance. This paper presents an efficient quantization and security enabled compressive video codec (QSCVC) framework compatible for wireless multimedia sensor networks. QSCVC focuses on secured transmission of the quantized CS measurements using scrambling. The security is ensured by the key controlled scrambler and descrambler algorithm used at the encoder and decoder respectively. On an average saving percentage of the QSCVC framework with quantization exceeds 13.19% without quantization and it is also observed that an average data rate of 736 Kbps is achieved. It is also proved that there is a 97.54% reduction in transmission energy when compared with raw frame transmission.

Downlink Non-Orthogonal Multiple Access With Limited Feedback

In this paper, we analyze downlink non-orthogonal multiple access (NOMA) networks with limited feedback. Our goal is to derive appropriate transmission rates for rate adaptation and minimize outage probability of minimum rate for the constant-rate data service, based on distributed channel feedback information from receivers. We propose an efficient quantizer with variable-length encoding that approaches the best performance of the case where perfect channel state information is available everywhere. We prove that in the typical application with two receivers, the losses in the minimum rate and outage probability decay at least exponentially with the minimum feedback rate. We analyze the diversity gain and provide a sufficient condition for the quantizer to achieve the maximum diversity order. For NOMA with $K$ receivers where $K > 2$ , we solve the minimum rate maximization problem within an accuracy of $\epsilon $ in time complexity of $O\left ({K\log \frac {1}{\epsilon }}\right )$ , and then, we apply the previously proposed quantizers for $K = 2$ to the case of $K > 2$ . Numerical simulations are presented to demonstrate the efficiency of our proposed quantizers and the accuracy of the analytical results.

Differential Feedback of Geometrical Mean Decomposition Precoder for Time-Correlated MIMO Systems

For a time-correlated channel, we consider the differential feedback of the geometrical mean decomposition (GMD) precoder, which is known to be optimal for a number of criteria. When the channel varies slowly, we can expect the optimal GMD precoders of consecutive channel uses to be close. We consider the feedback of the so-called differential precoder and show that it lies in a neighborhood of the identity matrix using matrix perturbation theory. Furthermore, the radius of the neighborhood is shown to be proportional to a time-correlation parameter. Such a characterization allows for subsequent efficient quantization of the differential precoder. We show how to design codewords for the prescribed radius by perturbing the identity matrix and applying QR decomposition on the perturbed matrix. Simulations are given to demonstrate that, with a small feedback rate, the performance of the proposed differential GMD comes close to the case when perfect channel state information is available to the transmitter for slowly time-varying channels like those in indoor and microcellular environments.

Online Variable Coding Length Product Quantization for Fast Nearest Neighbor Search in Mobile Retrieval

Quantization methods are crucial for efficient nearest neighbor search in many applications such as image, music, or product search. As mobile devices are becoming increasingly more popular, the quantization methods on mobile devices are more important, because a large portion of the search queries are becoming performed on mobile devices. One important characteristic of the communication on mobile devices is the inherent unreliability of their communication channels. In order to adapt the quality changes of the communication channels, we need to change the coding length of the quantization accordingly. The existing quantization methods use fixed-length codebooks, and it is expensive to retrain another codebook with different coding length. In this paper, we propose a novel variable length product quantization framework that consists of a set of fast universal scalar quantizers. The framework is capable of producing variable length quantization without retraining the codebook. Each data vector is transformed into a new space to reduce the correlation across dimensions. A proper number of bits is allocated to represent the scalar component in each dimension according to the given coding length. For each component, we estimate its probability density function (PDF) and design an efficient universal scalar quantizer based on the PDF and the allocated bits. To reduce distortion, we learn a Gaussian mixture model for the data. The experimental results show that, compared to state-of-the-art product quantization methods, our approach can construct the codebooks online for variable coding lengths and achieve the comparable performance.

Efficient quantization of tunable helix phase plates.

Helix phase plates are used in a variety of applications from optical trapping to astronomy. Tunable helix phase plates based on the Alvarez-Lohmann principle allow variation of the topological charge of the helix by rotating the phase plates with respect to each other around the optical axis. Current designs generate an undesired inverse phase in the section determined by the rotation angle. We present tunable phase plates that use a special quantization to maintain a uniform phase over the tuning range, suppressing the undesired part. As one benefit, the efficiency of the elements is increased over the whole tuning range.