Vector Quantizer Design Research Articles

Deterministic Annealing for Entropy-Constrained Vector Quantizer Design

Typical vector quantizer (VQ) design algorithms, such as generalized Lloyd methods, are susceptible to local minima. In addition, for entropy-constrained VQ, it is not obvious what the right number of codevectors is for any given source and target rate. One method that has shown promising results for solving both problems is deterministic annealing, where randomness is added to the encoding process, the uncertainty of which is gradually reduced until a deterministic coder is found. This method has been used for fixed-rate and entropy-constrained unstructured VQ, as well as for a variety of structurally constrained VQs. For the unstructured entropy-constrained case, we generalize the existing methods to include new cost functions, and we argue that, depending on the chosen cost function, some splitting criteria are more suitable than others. We give analytical advantages and disadvantages to several cost functions and several splitting criteria, then give performance evaluations by testing on synthetic and natural sources.

A G 2LA vector quantization for image data coding

In this paper, based on the Grey theory, a novel measurement method in a large volume and high dimension of information system is proposed for vector quantization (VQ) design and applied to image data coding. In the VQ coding procedure, it is often needs several epochs of clustering and always fails to obtain a better codebook; for instance, the well-known generalized Lloyd algorithm (GLA) easily traps into suboptimal codebook and does not have the ability to locate an optimal codebook during any clustering iteration with a random initial codebook. Hence, we propose a G 2LA design to solve heavy times of clustering procedure and at least to gain the best suboptimal codebook. In order to avoid edge degradation, firstly, the new selection of initial codevectors is adopted as the fast grey vector quantization (FGVQ) procedure which chooses nonhomogeneous vectors from a large volume image data. Then extending the GLA to G 2LA method by utilizing the measurement of grey relational analysis (GRA) which depends on the effect of relative objective and initial codevectors of FGVQ to obtain a better representative codebook. Experiment results show that at the same bit rate the G 2LA has not only the quickly convergence time but also high quality reconstructed image than traditional GLA technique with Euclidean distance measure, especially in high dimension and a large volume data system.

Optimal piecewise linear radial compression function for Laplacian source

AbstractIn order to achieve a quantisation technique that has low implementation complexity and performance arbitrarily close to that of Laplacian source optimal vector quantisation, a model of optimal geometric piecewise uniform cubic lattice vector quantiser will be proposed. It will be shown that the presented design procedure is not only much easier for realisation than the procedure of optimal vector quantiser design, but also gives the signal to quantisation noise ratio which in some cases differs from that of the optimal vector quantisation for 0.05 dB and less. It will also be demonstrated that the optimal geometric piecewise uniform vector quantiser model can be described with the optimal piecewise linear radial compression function. Further, since the optimal geometric piecewise uniform lattice vector quantisation in the case of large enough subregions number has performances identical to those of optimal vector quantisation, the optimal nonlinear radial compression function for Laplacian source will also be derived. Copyright © 2008 John Wiley & Sons, Ltd.

Index assignment for 3-description lattice vector quantization based on A2 lattice

Multiple description lattice vector quantization (MDLVQ) is an effective technique to realize multiple description coding. In this paper, we investigate a 3-description lattice vector quantization (LVQ) design using A 2 lattice, which finds a 3-tuple combination of sublattice points to represent each fine lattice point, based on two principles that each edge of the triangle formed by the 3-tuple points needs to be as short as possible, and the gravity center of the triangle is as close as possible to the fine lattice point. Following a delicate sublattice partition, a well-designed construction and mapping of each fine lattice point to a 3-tuple of sublattice points is developed to minimize the side distortion. The proposed index assignment is shown to achieve smaller distortion (up to 0.58 dB for Gaussian source) than some other 3-description index assignments. Compared with a 2-description LVQ, the 3-description LVQ achieves better expected rate-distortion performance in most cases of high description loss rates, while it also exhibits more graceful coding results with significantly smaller side distortions. Numerical analysis on Gaussian source and the simulations on image source validate the effectiveness of our proposed index assignment scheme.

Transmit beamforming in multiple-antenna systems with finite rate feedback: a VQ-based approach

This paper investigates quantization methods for feeding back the channel information through a low-rate feedback channel in the context of multiple-input single-output (MISO) systems. We propose a new quantizer design criterion for capacity maximization and develop the corresponding iterative vector quantization (VQ) design algorithm. The criterion is based on maximizing the mean-squared weighted inner product (MSwIP) between the optimum and the quantized beamforming vector. The performance of systems with quantized beamforming is analyzed for the independent fading case. This requires finding the density of the squared inner product between the optimum and the quantized beamforming vector, which is obtained by considering a simple approximation of the quantization cell. The approximate density function is used to lower-bound the capacity loss due to quantization, the outage probability, and the bit error probability. The resulting expressions provide insight into the dependence of the performance of transmit beamforming MISO systems on the number of transmit antennas and feedback rate. Computer simulations support the analytical results and indicate that the lower bounds are quite tight

Individual Convergence Rates in Empirical Vector Quantizer Design

We consider the rate of convergence of the expected distortion redundancy of empirically optimal vector quantizers. Earlier results show that the mean-squared distortion of an empirically optimal quantizer designed from n independent and identically distributed (i.i.d.) source samples converges uniformly to the optimum at a rate of O(1//spl radic/n), and that this rate is sharp in the minimax sense. We prove that for any fixed distribution supported on a given finite set the convergence rate is O(1/n) (faster than the minimax lower bound), where the corresponding constant depends on the source distribution. For more general source distributions we provide conditions implying a little bit worse O(logn/n) rate of convergence. Although these conditions, in general, are hard to verify, we show that sources with continuous densities satisfying certain regularity properties (similar to the ones of Pollard that were used to prove a central limit theorem for the code points of the empirically optimal quantizers) are included in the scope of this result. In particular, scalar distributions with strictly log-concave densities with bounded support (such as the truncated Gaussian distribution) satisfy these conditions.

Reduced-Complexity Deterministic Annealing for Vector Quantizer Design

This paper presents a reduced-complexity deterministic annealing (DA) approach for vector quantizer (VQ) design by using soft information processing with simplified assignment measures. Low-complexity distributions are designed to mimic the Gibbs distribution, where the latter is the optimal distribution used in the standard DA method. These low-complexity distributions are simple enough to facilitate fast computation, but at the same time they can closely approximate the Gibbs distribution to result in near-optimal performance. We have also derived the theoretical performance loss at a given system entropy due to using the simple soft measures instead of the optimal Gibbs measure. We use the derived result to obtain optimal annealing schedules for the simple soft measures that approximate the annealing schedule for the optimal Gibbs distribution. The proposed reduced-complexity DA algorithms have significantly improved the quality of the final codebooks compared to the generalized Lloyd algorithm and standard stochastic relaxation techniques, both with and without the pairwise nearest neighbor (PNN) codebook initialization. The proposed algorithms are able to evade the local minima and the results show that they are not sensitive to the choice of the initial codebook. Compared to the standard DA approach, the reduced-complexity DA algorithms can operate over 100 times faster with negligible performance difference. For example, for the design of a 16-dimensional vector quantizer having a rate of 0.4375 bit/sample for Gaussian source, the standard DA algorithm achieved 3.60 dB performance in 16 483 CPU seconds, whereas the reduced-complexity DA algorithm achieved the same performance in 136 CPU seconds. Other than VQ design, the DA techniques are applicable to problems such as classification, clustering, and resource allocation.

Novel vector quantiser design using reinforced learning as a pre-process

This paper presents the development and evaluation of a new approach toward the design of optimised codebooks by vector quantisation (VQ). A strategy of reinforced learning (RL) is proposed which exploits the advantages offered by fuzzy clustering algorithms, competitive learning and knowledge of training vector and codevector configurations. RL is used as a pre-process before using a conventional VQ algorithm such as the generalised Lloyd algorithm (GLA) or the fuzzy k-means (FKM) algorithm. At each iteration of RL, codevectors move intelligently and intentionally toward an improved optimum codebook design. This is distinct from simulated annealing (SA) and genetic algorithm (GA) techniques in which a random variation is introduced in the movement of the codevectors. The new strategy reduces the possibility that, in the final design, codevectors will be overcrowded in the high density distribution regions of the training vector space, and also that too few codevectors will be settled in low density regions. Experiments demonstrate that this results in a more effective representation of the training vectors by the codevectors and that the final codebook is nearer to the optimal solution in applications such as image compression. It has been found that GLA and FKM yield improved quality of codebook design in this application when RL is used as a pre-process. The investigations have also indicated that RL is insensitive to the selection of both the initial codebook and a learning rate control parameter, which is the only additional parameter introduced by RL from the standard FKM.

Reduced-complexity deterministic annealing for vector quantizer design

This paper presents a reduced-complexity deterministic annealing (DA) approach for vector quantizer (VQ) design by using soft information processing with simplified assignment measures. Low-complex...

On Finite-State Vector Quantization for Noisy Channels

Finite-state vector quantization (FSVQ) over a noisy channel is studied. A major drawback of a finite-state decoder is its inability to track the encoder in the presence of channel noise. In order to overcome this problem, we propose a nontracking decoder which directly estimates the code vectors used by a finite-state encoder. The design of channel-matched finite-state vector quantizers for noisy channels, using an iterative scheme resembling the generalized Lloyd algorithm, is also investigated. Simulation results based on encoding a Gauss-Markov source over a memoryless Gaussian channel show that the proposed decoder exhibits graceful degradation of performance with increasing channel noise, as compared with a finite-state decoder. Also, the channel-matched finite-state vector quantizers are shown to outperform channel-optimized vector quantizers having the same vector dimension and rate. However, the nontracking decoder used in the channel-matched finite-state quantizer has a higher computational complexity, compared with a channel-optimized vector-quantizer decoder. Thus, if they are allowed to have the same overall complexity (encoding and decoding), the channel-optimized vector quantizer can use a longer encoding delay and achieve similar or better performance. Finally, an example of using the channel-matched finite-state quantizer as a backward-adaptive quantizer for nonstationary signals is also presented.

Multiresolution Vector Quantization

Multiresolution source codes are data compression algorithms yielding embedded source descriptions. The decoder of a multiresolution code can build a source reproduction by decoding the embedded bit stream in part or in whole. All decoding procedures start at the beginning of the binary source description and decode some fraction of that string. Decoding a small portion of the binary string gives a low-resolution reproduction; decoding more yields a higher resolution reproduction; and so on. Multiresolution vector quantizers are block multiresolution source codes. This paper introduces algorithms for designing fixed- and variable-rate multiresolution vector quantizers. Experiments on synthetic data demonstrate performance close to the theoretical performance limit. Experiments on natural images demonstrate performance improvements of up to 8 dB over tree-structured vector quantizers. Some of the lessons learned through multiresolution vector quantizer design lend insight into the design of more sophisticated multiresolution codes.

Genetic channel-optimized vector quantizer design for burst error channels

This paper presents a novel vector quantizer (VQ) design algorithm optimized to a burst error channel (BEC) for robust communication. The Gilbert–Elliot model is used to describe the BEC. Based on the model, the objective of this algorithm is to minimize the average distortion when the BEC is in the normal state of operation, while maintaining a minimum fidelity when the BEC is in the undesirable state. In the algorithm, an iterative design procedure is first derived for obtaining a local optimal solution to the problem. A novel genetic scheme is then proposed for attaining a near global optimal performance. Numerical results show that, when delivering information over the BEC, the algorithm significantly outperforms the VQ techniques optimizing the design only to the simple binary symmetric channels.

Robust vector qantization for burst error channels

This paper presents a novel vector quantizer (VQ) design algorithm for a burst error channel (BEC). The Gilbert‐Elliot model is used to describe the BEC. Based on the model, the objective of this algorithm is to minimize the average distortion when the BEC is in a normal state of operation, while maintaining a minimum fidelity when the BEC is in the undesirable state. Both the encoding and decoding operations of the VQ depend on the channel state observations. An optimal iterative design procedure is then derived for the general case where only noisy channel state information is available. Numerical results show that, when delivering information over the BEC's, the algorithm significantly outperforms VQ techniques optimizing the design only to the simple binary symmetric channels.

An Enhanced Generalized Lloyd Algorithm

Vector quantization is an effective means of data compression for it maps a set of real numbers into a single integer. In this letter an enhanced generalized Lloyd algorithm (GLA) for vector quantizer design is presented. Experiment results show that the proposed scheme outperforms the conventional GLA and recently reported modification of the K-means algorithm. Moreover, the achievement in coding performance is not accompanied by significant increase in computational complexity.

Joint Design of Channel-Optimized Multistage Vector Quantizer

A novel channel-optimized multistage vector quantization (COMSVQ) codec is presented in which the stage codebooks are jointly designed. The proposed codec uses a signal source and channel-dependent distortion measure to encode line spectral frequencies derived from segments of a speech signal. Simulation results are provided to demonstrate the consistent reduction in the spectral distortion obtained using the proposed codec as compared to the conventional sequentially designed channel-matched multistage vector quantizer.

문맥 독립 화자인식을 위한 공간 분할 벡터 양자기 설계

In this paper, we study the enhancement of VQ (Vector Quantization) design for text independent speaker recognition. In a concrete way, we present a non-iterative method which makes a vector quantization codebook and this method performs non-iterative learning so that the computational complexity is epochally reduced The proposed Classified Space VQ (CSVQ) design method for text Independent speaker recognition is generalized from Semi-noniterative VQ design method for text dependent speaker recognition. CSVQ contrasts with the existing desiEn method which uses the iterative learninE algorithm for every traininE speaker. The characteristics of a CSVQ design is as follows. First, the proposed method performs the non-iterative learning by using a Classified Space Codebook. Second, a quantization region of each speaker is equivalent for the quantization region of a Classified Space Codebook. And the quantization point of each speaker is the optimal point for the statistical distribution of each speaker in a quantization region of a Classified Space Codebook. Third, Classified Space Codebook (CSC) is constructed through Sample Vector Formation Method (CSVQ1, 2) and Hyper-Lattice Formation Method (CSVQ 3). In the numerical experiment, we use the 12th met-cepstrum feature vectors of 10 speakers and compare it with the existing method, changing the codebook size from 16 to 128 for each Classified Space Codebook. The recognition rate of the proposed method is 100% for CSVQ1, 2. It is equal to the recognition rate of the existing method. Therefore the proposed CSVQ design method is, reducing computational complexity and maintaining the recognition rate, new alternative proposal and CSVQ with CSC can be applied to a general purpose recognition.

Multiple description quantization by deterministic annealing

The design of vector quantizers for diversity-based communication over two or more channels of possibly differing capacities and failure probabilities, is considered. The crucial dependence of current design techniques on initialization, especially of index assignment, is well recognized. Instead, we propose to pursue a deterministic annealing approach which is independent of initialization, does not assume any prior knowledge of the source density, and avoids many poor local minima of the cost surface. The approach consists of iterative optimization of a random encoder at gradually decreasing levels of randomness as measured by the Shannon entropy. At the limit of zero entropy, a hard multiple description (MD) quantizer is obtained. This process is directly analogous to annealing processes in statistical physics. Via an alternative derivation, we show that it may also be interpreted as approximating the minimum rate sums among points on the convex hull of the MD achievable rate-distortion region of El Gamal and Cover, subject to constraints on the sizes of the reproduction alphabets. To illustrate the potential of our approach, we present simulation results that show substantial performance gains over existing design techniques.

문맥종속 화자인식을 위한 준비반복 벡터 양자기 설계 알고리즘

In this paper, we study the enhancement of VQ (Vector Quantization) design for text dependent speaker recognition. In a concrete way, we present the non-Iterative method which makes a vector quantization codebook and this method Is nut Iterative learning so that the computational complexity is epochally reduced. The proposed semi-noniterative VQ design method contrasts with the existing design method which uses the iterative learning algorithm for every training speaker. The characteristics of a semi-noniterative VQ design is as follows. First, the proposed method performs the iterative learning only for the reference speaker, but the existing method performs the iterative learning for every speaker. Second, the quantization region of the non-reference speaker is equivalent for a quantization region of the reference speaker. And the quantization point of the non-reference speaker is the optimal point for the statistical distribution of the non-reference speaker In the numerical experiment, we use the 12th met-cepstrum feature vectors of 20 speakers and compare it with the existing method, changing the codebook size from 2 to 32. The recognition rate of the proposed method is 100% for suitable codebook size and adequate training data. It is equal to the recognition rate of the existing method. Therefore the proposed semi-noniterative VQ design method is, reducing computational complexity and maintaining the recognition rate, new alternative proposal.

Tree-structured multiple description coding

Multiple description coding aims at transmitting two mutually refinable descriptions of a source on two different channels. It is successfully applied to transmission of multimedia signals on diversity systems or packet-switched networks. We present simple algorithms for the design of multiply-descriptive tree-structured vector quantizers with linear encoding complexity. These algorithms allow for both the rate and distortion constraints in a natural way and are shown to yield quantizers that are competitive with full-search codes in terms of rate-distortion performance.

Predictive vector quantizer design using deterministic annealing

A new approach is proposed for predictive vector quantizer (PVQ) design, which is inherently probabilistic, and is based on ideas from information theory and analogies to statistical physics. The approach effectively resolves three longstanding fundamental shortcomings of standard PVQ design. The first complication is due to the PVQ prediction loop, which has a detrimental impact on the convergence and the stability of the design procedure. The second shortcoming is due to the piecewise constant nature of the quantizer function, which makes it difficult to optimize the predictor with respect to the overall reconstruction error. Finally, a shortcoming inherited from standard VQ design is the tendency of the design algorithm to terminate at a locally, rather than the globally, optimal solution. We propose a new PVQ design approach that embeds our previous asymptotic closed-loop (ACL) approach within a deterministic annealing (DA) framework. The overall DA-ACL method profits from its two main components in a complementary way. ACL is used to overcome the first difficulty and offers the means for stable quantizer design as it provides an open-loop design platform, yet allows the PVQ design algorithm to asymptotically converge to optimization of the closed-loop performance objective. DA simultaneously mitigates or eliminates the remaining design shortcomings. Its probabilistic framework replaces hard quantization with a differentiable expected cost function that can be jointly optimized for the predictor and quantizer parameters, and its annealing schedule allows the avoidance of many poor local optima. Substantial performance gains over traditional methods have been achieved in the simulations.