Lloyd-Max Quantization Research Articles

Distribution Agnostic Symbolic Representations for Time Series Dimensionality Reduction and Online Anomaly Detection

Due to the importance of the lower bounding distances and the attractiveness of symbolic representations, the family of symbolic aggregate approximations (SAX) has been used extensively for encoding time series data. However, typical SAX-based methods rely on two restrictive assumptions; the Gaussian distribution and equiprobable symbols. This paper proposes two novel data-driven SAX-based symbolic representations, distinguished by their discretization steps. The first representation, oriented for general data compaction and indexing scenarios, is based on the combination of kernel density estimation and Lloyd-Max quantization to minimize the information loss and mean squared error in the discretization step. The second method, oriented for high-level mining tasks, employs the Mean-Shift clustering method and is shown to enhance anomaly detection in the lower-dimensional space. Besides, we verify on a theoretical basis a previously observed phenomenon of the intrinsic process that results in a lower than the expected variance of the intermediate piecewise aggregate approximation. This phenomenon causes an additional information loss but can be avoided with a simple modification. The proposed representations possess all the attractive properties of the conventional SAX method. Furthermore, experimental evaluation on real-world datasets demonstrates their superiority compared to the traditional SAX and an alternative data-driven SAX variant.

HEMP: High-order entropy minimization for neural network compression

We formulate the entropy of a quantized artificial neural network as a differentiable function that can be plugged as a regularization term into the cost function minimized by gradient descent. Our formulation scales efficiently beyond the first order and is agnostic of the quantization scheme. The network can then be trained to minimize the entropy of the quantized parameters, so that they can be optimally compressed via entropy coding. We experiment with our entropy formulation at quantizing and compressing well-known network architectures over multiple datasets. Our approach compares favorably over similar methods, enjoying the benefits of higher order entropy estimate, showing flexibility towards non-uniform quantization (we use Lloyd-max quantization), scalability towards any entropy order to be minimized and efficiency in terms of compression. We show that HEMP is able to work in synergy with other approaches aiming at pruning or quantizing the model itself, delivering significant benefits in terms of storage size compressibility without harming the model’s performance.

Classification of brain tumors and auto-immune disease using ensemble learning

In this era of rapid technological advancement, computer vision in medical analysis and diagnosis has a profound significance in detecting and classifying anomalies or diseases in the human body. This study proposes an ensemble learning method to classify brain tumors or neoplasms (i.e. glioma, meningioma, pituitary adenoma) and auto-immune disease lesion (i.e. multiple sclerosis) using magnetic resonance imaging (MRI) of brain tumors and multiple sclerosis patients. The method includes pre-processing, feature extraction, feature selection, and classification. The pre-processing phase uses region of interest (ROI) of both tumor and lesion, Collewet normalization, and Lloyd-max quantization. The base learner is designed with a support vector machine (SVM) classifier and prediction model with majority voting. Our proposed system operates well in contrast to other state-of-the-art methods, with the weighted sensitivity, specificity, precision, and accuracy of 97.5%, 98.838%, 98.011%, and 98.719%, respectively. Experimental results also show that the overall training and testing accuracy of our proposed model are 97.957% and 97.744% respectively. The findings may infer an unprecedented step for detecting the presence of lesions coexisting with the tumors in neuro-medicine diagnosis.

On the Structure and Realization of Real-Time Coding and Estimation Over Noiseless Channel

This paper presents the optimal structural analysis and realization scheme of real-time coding and estimation for linear discrete-time system over rate-constrained noiseless channel. The mean-squared error (MSE) is adopted as the estimation performance criterion. Without any loss of performance, the optimal real-time coding and estimation functions can be represented in recursive forms, which may reduce the potential computational complexity and storage burden. Furthermore, we put forward a recursive realization scheme via standard Kalman filter and vector Lloyd-Max quantization, which is also shown to be structurally equivalent to the established structural results. Moreover, to synchronize the encoder-decoder pair instantaneously, there are two synchronizers respectively arranged at the encoder and decoder sides, which utilize the common information between the encoder and the decoder to synchronize the encoding and decoding operations. Our proposed realization scheme is demonstrated with simulation results, and the designed algorithm performs well even for very low rate region.

A novel high payload steganography scheme based on absolute moment block truncation coding

In this paper, we propose an efficient steganography method in the compressed codes of absolute moment block truncation coding (AMBTC). Many recent related schemes focus on implementing reversible data hiding in compressed AMBTC bit stream. However, the reconstructed image of AMBTC is already lossy and the strict reversibility severely limits embedding capacity. Due to the simplicity and regularity of AMBTC codes, implementing irreversible hiding scheme causes very slight loss visual distortion of reconstructed image in exchange of significant improve in embedding capacity. In proposed scheme, smoothness of AMBTC compressed trio is firstly detected, which is then indicated by substituting the LSB of high quantity level with flag bit. For smooth trios, the differences between both quantity levels are firstly encoded by Huffman coding and then concatenated with secret data to generate modified low quantity levels. Meanwhile, all bits in bit planes of smooth trios are substituted with secret data as well. For complex trio, secret bits are only embedded into quantity levels, which is similar to smooth trio except for the differences are encoded by Lloyd-Max quantization. Experimental results indicate that proposed scheme outperforms prior methods both in imperceptivity and embedding capacity, which confirms the effectiveness and superiority of our work.

Finite-state Markov wireless channel modeling for railway tunnel environments

In recent years, high-speed railways (HSRs) have developed rapidly with a high transportation capacity and high comfort level. A tunnel is a complex high-speed rail terrain environment. It is very important to establish an accurate channel propagation model for a railway tunnel environment to improve the safety of HSR operation. In this paper, a method for finite-state Markov chain (FSMC) channel modeling with least squares fitting based on non-uniform interval division is proposed. First, a path loss model is obtained according to measured data. The communication distance between the transmitter and receiver in the tunnel is non-uniformly divided into several large non-overlapping intervals based on the path loss model. Then, the Lloyd-Max quantization method is used to determine the threshold of the signal-to-noise ratio (SNR) and the channel state quantization value and obtain the FSMC state transition probability matrix. Simulation experiments show that the proposed wireless channel model has a low mean square error (MSE) and can accurately predict the received signal power in a railway tunnel environment.

Invited Paper] Digital RoHR with Lloyd-Max Quantization for Distributed Collaborative MIMO-OFDM Reception

Invited Paper] Digital RoHR with Lloyd-Max Quantization for Distributed Collaborative MIMO-OFDM Reception

Real Time Coding and Estimation of Linear Discrete Time System Over Networks

This study deals with the real time coding and estimation of linear discrete time scalar system over communication networks. With the mean-squared error (MSE) distortion criterion, the information rate distortion function describing the performance limit of the coding-estimation system is analyzed and discussed. To achieve near instantaneous encoding and decoding, an asymptotic design scheme has been presented as a realization of real time coding-estimation system. The outputs of standard Kalman filter relying on unquantized observations are encoded using the Lloyd-Max quantization rules and transmitted over the noiseless channel. The decoder side runs the corresponding decoding and reconstruction algorithms to produce the optimal real time state estimate of system. To synchronize the encoder and decoder, a double-predictor regime on the updating rules of the encoder-decoder pair is proposed, and it does not require any feedback information. The rate distortion function of the proposed scheme is derived and when comparing with the information theoretical lower bound, there is only a factor discrepancy related to the quantization rules. The rate distortion performance results of various design schemes are compared and demonstrated with numerical simulations.

Performance improvement of wireless secret key generation with colored noise for IoT

SummaryAchieving security in the Internet of things (IoT) networks by generating symmetric keys from the wireless channel parameters like received signal strength (RSS) is a promising approach. Despite the easy acquisition of the RSS signal, RSS‐based security is less explored for IoT. In this work, we analyze the performance of RSS‐based wireless physical layer key generation with correlated colored noise components and proposed a low complexity filtering approach to improve the performance for the IoT network. We started with providing a survey of various recent researches related to RSS‐based key generation and also discussed correlated colored noise components with a few of the recent works considering them. Further, we analyze various colored noise components in the time domain by the Allan variance and Ljung‐Box test. Furthermore, we develop a key generation model and proposed a moving window averaging‐based filtering followed by Lloyd max quantization to improve the BDR performance, degraded due to the presence of correlated colored noise components. The simulation results show that the proposed preprocessing technique has a considerable improvement in the BDR performance, and the keys generated have sufficient randomness, which is verified by NIST test.

On sparsity of eigenportfolios to reduce transaction cost

Purpose Transaction cost becomes significant when one holds many securities in a large portfolio where capital allocations are frequently rebalanced due to variations in non-stationary statistical characteristics of the asset returns. The purpose of this paper is to employ a sparsing method to sparse the eigenportfolios, so that the transaction cost can be reduced and without any loss of its performance. Design/methodology/approach In this paper, the authors have designed pdf-optimized mid-tread Lloyd-Max quantizers based on the distribution of each eigenportfolio, and then employed them to sparse the eigenportfolios, so those small size orders may usually be ignored (sparsed), as the result, the trading costs have been reduced. Findings The authors find that the sparsing technique addressed in this paper is methodic, easy to implement for large size portfolios and it offers significant reduction in transaction cost without any loss of performance. Originality/value In this paper, the authors investigated the performance the sparsed eigenportfolios of stock returns in S&P500 Index. It is shown that the sparsing method is simple to implement and it provides high levels of sparsity without causing PNL loss. Therefore, transaction cost of managing a large size portfolio is reduced by employing such an efficient sparsity method.

Data Compression Scheme of Fronthaul Network Based on LTE

As the long term evolution (LTE) mobile users and transmission data increase, the load of the fronthaul network increases. In order to control the consumption of optical fiber resources, and prevent congestion under the premise of increasing data transmission, it is necessary to compress the data of the fronthaul network. In this paper, a data compression scheme of LTE-based fronthaul network is proposed. According to the characteristics of LTE baseband signals, discrete sine transform (DST) is applied to the time domain signals, and the transformed coefficients are partitioned according to the energy concentration characteristics. Bit allocation is performed in different blocks, and the coefficients of each block are quantized by Lloyd-Max quantizer. Finally, Huffman coding is carried out to improve the compression ratio under the condition that the error is allowed. The simulation results show that the proposed data compression scheme has good performance in both compression ratio (CR) and error vector magnitude (EVM).

An Efficient And Adaptive Compression Algorithm Based On DCT Transform For GPR Raw Data

Ground Penetrating Radar(GPR) creates large amounts of data which causes serious problems for storage and transmission. In this paper, a novel compression algorithm for GPR raw data based on DCT transform and inter-frame difference is proposed. The algorithm first conducts DCT transform on the GPR raw data, and differentiates the adjacent frame data in DCT domain to reduce the variance and dynamic range. Then the differential data is evenly divided into several blocks, and different bit Lloyd-Max quantizers are selected to quantify different sub-block data through an adaptive bit allocation mechanism, thereby obtaining a higher compression gain. The compression experiments with satellite-borne GPR, aircraft-borne GPR and vehicle-borne GPR raw data confirmed the effectiveness of the compression algorithm.

Forward Adaptive Laplacian Source Coding Based on Restricted Quantization

A novel solution for Laplacian source coding based on three-level quantization is proposed in this paper. The restricted three-level quantizer is designed by assuming the restricted Laplacian distribution of the input signal. Quantizer and Huffman encoder are jointly designed. Forward adaptive scheme was employed, where the adaptation to the signal variance (power) was performed on frame-by frame basis. We employ switched model that consists of two restricted quantizers having unequal support regions. The simulation results (measured as SQNR) of the proposed scheme with a switched restricted three-level quantizer are compared to the cases when it involves three-level unrestricted quantizer and the Lloyd-Max quantizers having N=2 and N=4 levels. It is shown that the proposed solution offers performance comparable to the one of N=4 levels Lloyd-Max’s baseline with large savings in bit rate, while outperforming two other baselines.DOI: http://dx.doi.org/10.5755/j01.itc.47.2.16670

Design of a Multi-mode Block Adaptive Quantizer for High-Resolution Synthetic Aperture Radar Imaging

An advanced synthetic aperture radar imaging system needs a multi-mode block adaptive quantizer (BAQ) for high-resolution image requirements. However, the Lloyd-Max quantizer inside the BAQ consists of non-uniform thresholds for comparison. Consequently, the arithmetic complexity grows exponentially if the BAQ output word-length becomes larger. In this work, a multi-mode BAQ is designed to support real-time compression for high sampling-frequency radar signals. Various output word-lengths are provided so that different compression ratios can be set depending on the applications. To solve the problem of exponential growth in complexity due to the large BAQ output word-lengths, we propose two strategies, input scaling and hybrid comparators. Significant saving in complexity is attained. From the synthesis result in 40 nm CMOS technology, the proposed design supports 14-bit ADC at 800-MHz sample frequency with output word-lengths of 2, 3, 4, 6, 8 bits. Its gate count is 45.6 K and power consumption is only 11.8 mW at 0.9 V.

On the Trade-Off Between Bit Depth and Number of Samples for a Basic Approach to Structured Signal Recovery From <inline-formula> <tex-math notation="LaTeX">$b$ </tex-math> </inline-formula>-Bit Quantized Linear Measurements

We consider the problem of recovering a high-dimensional structured signal from independent Gaussian linear measurements each of which is quantized to $b$ bits. The focus is on a specific method of signal recovery that extends a procedure originally proposed by Plan and Vershynin for one-bit quantization to a multi-bit setting. At the heart of this paper is a characterization of the optimal trade-off between the number of measurements $m$ and the bit depth per measurement $b$ given a total budget of $B = m \cdot b$ bits when the goal is to minimize the expected $\ell _{2}$ -error in estimating the signal. It turns out that the choice $b = 1$ is optimal for estimating the unit vector (direction) corresponding to the signal for any level of additive Gaussian noise before quantization as well as for a specific model of adversarial noise, whereas in a noiseless setting the choice $b = 2$ is optimal for estimating the direction and the norm (scale) of the signal. Moreover, Lloyd–Max quantization is shown to be an optimal quantization scheme with respect to $\ell _{2}$ -estimation error. Our analysis is corroborated by the numerical experiments showing nearly perfect agreement with our theoretical predictions. This paper is complemented by an empirical comparison to alternative methods of signal recovery. The results of that comparison point to a regime change depending on the noise level: in a low-noise setting, the approach under study falls short of more sophisticated competitors while being competitive in moderate- and high-noise settings.

Optimizing Quantization for Lasso Recovery

This letter is focused on quantized compressed sensing, assuming that Lasso is used for signal estimation. Leveraging recent work, we propose a constrained Lloyd–Max-like framework to optimize the quantization function in this setting, and show that when the number of observations is high, this method of quantization gives a significantly better recovery rate than standard Lloyd–Max quantization. We support our theoretical analysis with numerical simulations.

Quantize-and-Forward Strategy for Interleave-Division Multiple-Access Relay Channel

The quantize-and-forward (QF) strategy for the half-duplex interleave-division multiple-access relay channel (ID-MARC) is investigated in this paper with multiple users, single relay, and single destination. Different from the orthogonal MARC, multiple users in such an ID-MARC are distinguished by different interleavers. For the case that the relay node cannot recover the initial bits correctly, a denoising mutual-information preserving quantizer (DMIPQ) is proposed, which can remove the noise from the received signal while preserving the useful information. A suboptimal searching algorithm is designed to reduce the complexity of DMIPQ. The approximation of DMIPQ is further discussed to get an analytical solution. It is shown that the approximate DMIPQ is equivalent to the minimum mean square error (MMSE) Lloyd-max quantizer. Numerical results illustrate that the proposed DMIPQ outperforms the MMSE Lloyd-max quantizer. It can be also shown that the bit error rate of the ID-MARC (BER) with DMIPQ is close to its compress-and-forward (CF) bounds.

Efficient Quantization Method for Biometric Fingerprint Image Compression

The choice of quantization method and the requirement to achieve a trade-off between compressed image quality and degradation are very crucial in the overall performance of a lossy image compression algorithm. In this paper, uniform and non-uniform scalar quantization schemes of biometric fingerprint image were studied. Comparative analyses of non-uniform quantization methods were also conducted and these include dither-based quantization and the Lloyd-Max quantization methods. The quality of the quantized output fingerprint image was determined in terms of Signal-to-Quantization Noise Ratio (SQNR). The degree of distortion or quantization error was determined in terms of the Mean Square Quantization Error (MSQE). The non-uniform quantization method performed better than the uniform quantization method in terms of the SQNR and MSQE values. It was also found out that, the performance of dither-based non-uniform quantization on biometric fingerprint image is not as efficient as the LloydMax approach when the number of bits used in the quantization process increased. The results showed that the higher the number of bits used in the quantization process the higher the quality and the less the distortion in the resulting images.

Research on Lloyd-Max Quantizer with Two-Stage Otsu’s Method

Otsu’s method is often used in image segmentation applications such as defect detections, medical image diagnosis and object shape recognitions. However it is very time-consuming for multilevel segmentation. Lloyd-Max quantizer is a popular and efficient data compressor. Fundamentally, Otsu’s method and Lloyd-Max quantizer are equivalent to maximum a posteriori probability estimate. Applying them on multilevel image segmentation, we can find their segmented results over an image are very approximate, but Otsu’s method running in exhaustive search consumes more processing time than Lloyd-Max quantizer with iterative characteristics does. Thus, Lloyd-Max quantizer is strongly recommended as the fast and first-stage agent for Otsu’s method to find the optimal threshold values for image segmentation.

Fully Private Noninteractive Face Verification

Face recognition is one of the foremost applications in computer vision, which often involves sensitive signals; privacy concerns have been raised lately and tackled by several recent privacy-preserving face recognition approaches. Those systems either take advantage of information derived from the database templates or require several interaction rounds between client and server, so they cannot address outsourced scenarios. We present a private face verification system that can be executed in the server without interaction, working with encrypted feature vectors for both the templates and the probe face. We achieve this by combining two significant contributions: 1) a novel feature model for Gabor coefficients' magnitude driving a Lloyd-Max quantizer, used for reducing plaintext cardinality with no impact on performance; 2) an extension of a quasi-fully homomorphic encryption able to compute, without interaction, the soft scores of an SVM operating on quantized and encrypted parameters, features and templates. We evaluate the private verification system in terms of time and communication complexity, and in verification accuracy in widely known face databases (XM2VTS, FERET, and LFW). These contributions open the door to completely private and noninteractive outsourcing of face verification.