Vector Quantization Technique Research Articles

Estimation of the probability density function and a posteriori probability by neural networks, and applications to vowel recognition

AbstractFeed‐forward neural networks have been used for pattern recognition, because they have an ability to estimate a posteriori probability. This paper investigates the ability to estimate the a posteriori probability by using one‐dimensional Gaussian distributions, uniform distributions, their mixed distributions and real speech data, and applies the networks to speech recognition. Furthermore, the ability to estimate a probability density function of artificial data by using a vector quantization technique and neural networks and also to apply them to speech recognition also are investigated. Feed‐forward neural networks, radial basis function networks (RBF), Gaussian mixed distributions and multitemplate methods for speech recognition are compared. It is concluded that the vector quantization‐based RBF is the best in practice.

Sequential scalar quantization of color images

An efficient algorithm for color image quantization is proposed based on a new vector quantization technique that we call sequential scalar quantization. The scalar components of the 3-D color vector are individually quantized in a predetermined sequence. With this technique, the color palette is designed very efficiently, while pixel mapping is performed with no computation. To obtain an optimal allocation of quantization levels along each color coordinate, we appeal to the asymptotic quantization theory, where the number of quantization levels is assumed to be very large. We modify this theory to suit our application, where the number of quantization 1evels is typically small. To utilize the properties of the human visual system (HVS), the quantization is performed in a luminance-chrominance color space. A luminance-chrominance weighting is introduced to account for the greater sensitivity of the HVS to luminance than to chrominance errors. A spatial activity measure is also incorporated to reflect the increased sensitivity of the HVS to quantization errors in smooth image regions. The algorithm yields high-quality images and is significantly faster than existing quantization algorithms.

Edge-based subband VQ techniques for images and video

A key issue in subband coding is the efficient compression of the less informative but perceptually important upper frequency bands of the decomposed image. A new approach capable of effectively encoding the upper-bands is described. An intra-band vector quantization (VQ) technique is employed for compression of the base-band while the upper frequency bands are encoded by a hierarchical inter-band VQ method. The proposed inter-band vector quantization scheme exploits the redundancies that exist between pels of significant perceptual importance across the upper frequency bands of the same resolution. Such pels are observed to be at or around the edge-locations, displaying similar discontinuity behavior across the subbands. Therefore, an edge-detector is applied on the reconstructed base-band to extract the edge locations, and as a result no overhead information is transmitted to identify the position of these pels in the upper-bands. Furthermore, a residual subband, being the difference between the original base-band of each layer and its encoded version, is incorporated in the proposed inter-band VQ model. Thus, the proposed interband VQ scheme is simply the vector quantization of pixels across the upper-bands together with their corresponding residual band. Compression results are presented for both digital images and video sequences which demonstrate high subjective visual qualities.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Use of multiple vector quantisation for semicontinuous-HMM speech recognition

Although the continuous hidden Markov model (CHMM) technique seems to be the most flexible and complete tool for speech modelling. It is not always used for the implementation of speech recognition systems because of several problems related to training and computational complexity. Thus, other simpler types of HMMs, such as discrete (DHMM) or semicontinuous (SCHMM) models, are commonly utilised with very acceptable results. Also, the superiority of continuous models over these types of HMMs is not clear. The authors' group has previously introduced the multiple vector quantisation (MVQ) technique, the main feature of which is the use of one separated VQ codebook for each recognition unit. The MVQ technique applied to DHMM models generates a new HMM modelling (basic MVQ models) that allows incorporation into the recognition dynamics of the input sequence information wasted by the discrete models in the VQ process. The authors propose a new variant of HMM models that arises from the idea of applying MVQ to SCHMM models. These are SCMVQ-HMM (semicontinuous multiple vector quantisation HMM) models that use one VQ codebook per recognition unit and several quantisation candidates for each input vector. It is shown that SCMVQ modelling is formally the closest one to CHMM, although requiring even less computation than SCHMMs. After studying several implementation issues of the MVQ technique. Such as which type of probability density function should be used, the authors show the superiority of SCMVQ models over other types of HMM models such as DHMMs, SCHMMs or the basic MVQs.

A constrained vector quantization scheme for real-time codebook retransmission

One of the major obstacles to the use of vector quantization techniques in practice is the huge design complexity. Huang and Huang (1991, 1992, 1993) and Huang (1991) developed a tree-structured polytopal vector quantization scheme referred to as the principal component vector quantization algorithm (PCVQA). It has been shown that the design complexity of PCVQA is only linearly proportional to the codebook size. The present paper examines various techniques to further reduce the implementational complexity for PCVQA so that the algorithm is suitable for real-time codebook retransmission. In addition, an extension of PCVQA is developed to exploit both inter-block and intra-block redundancy and to deal with high definition images.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Microphone arrays and speaker identification

Hands-free operation of speech processing equipment is sometimes desired so that the user is unencumbered by hand-held or body-worn microphones. This paper explores the use of array microphones to capture speech under adverse acoustic conditions, and provide input to a system for automatic speaker identification. The system is evaluated using reverberated speech signals, generated by a computer model of room acoustics and transduced by different simulated microphone-arrays. For comparison, the system is also evaluated using close-talking microphone input. It is found that 2-D matched-filter microphone arrays are capable of producing high speaker identification scores in a hostile acoustic environment, such as multipath distortion and competing noise sources. The paper also explores the influence of vector quantization techniques, codebook size, and order of cepstrum coefficients on the performance of the speaker identification system. >

Global Optimization Using the Multistart Method

Multistart is a stochastic global optimization method for finding the global optimum of highly nonlinear mechanical problems. In this paper we introduce and develop a variant of the multistart method in which a fraction of the sample points in the feasible region with smallest function value are clustered using the Vector Quantization technique. The theories of lattices and sphere packing are used to define optimal lattices. These lattices are optimal with respect to quantization error and are used as code points for vector quantization. The implementation of these ideas has resulted in the VQ-multistart algorithm for finding the global optimum with substantial reductions in both the incore memory requirements and the computation time. We solve several mathematical test problems and a mechanical optimal design problem using the VQ-multistart algorithm.

Enhancements to DTW and VQ decision algorithms for speaker recognition

Dynamic Time Warping (DTW) and Vector Quantisation (VQ) techniques have been applied with considerable success to speaker verification. It is standard practice to use these techniques to calculate a single distance score, and threshold this value to produce a verification decision. In this paper we examine applying a statistical weighting to a number of parameters extracted using the DTW warp path and VQ decision mechanisms. Results are presented which show that the additional parameters extracted encode further speaker specific information, and can be used to improve upon the speaker verification performance of the baseline systems. The application of a distance normalisation technique, which involves comparing DTW or VQ scores for the claimed identity against other speakers, is also investigated. Speaker verification results for baseline and enhanced DTW and VQ systems are reported for a population of 42 speakers.

Principal Component Vector Quantization

This paper proposes a vector quantization technique which is referred to here as the Principal Component Vector Quantization algorithm. It also proposes an artificial neural network with an on-line learning algorithm for codebook design, encoding, and decoding. For unknown distribution cases, the proposed technique can be implemented directly without calling for repeated use of training sequences. The main features of the proposed technique include (i) codebook design complexity is only linearly proportional to the codebook size; (ii) encoding complexity is in the order of that of typical tree structured schemes. (iii) The proposed scheme is suitable for coding video sequences which may involve nonstationary training sequences. Note that the same neural network structure can be used to implement general full-search vector quantization algorithms, featuring constant encoding time which is independent of the codebook size.

Novel interpolative vector quantisation technique for image coding

An innovative technique for the interpolative vector quantisation (IVQ) of an image is developed. The technique employs a new split-and-merge algorithm which selects a near-to-optimal set of sampling points for the IVQ. Experimental results show that, under the same bit rate, over 0.5 dB improvement is achieved by the proposed method when compared with the results obtained by the traditional fixed sampling point interpolation in terms of peak signal-to-noise ratio (PSNR).

Implementation of self-organizing neural networks for visuo-motor control of an industrial robot

The implementation of two neural network algorithms for visuo-motor control of an industrial robot (Puma 562) is reported. The first algorithm uses a vector quantization technique, the ;neural-gas' network, together with an error correction scheme based on a Widrow-Hoff-type learning rule. The second algorithm employs an extended self-organizing feature map algorithm. Based on visual information provided by two cameras, the robot learns to position its end effector without an external teacher. Within only 3000 training steps, the robot-camera system is capable of reducing the positioning error of the robot's end effector to approximately 0.1% of the linear dimension of the work space. By employing adaptive feedback the robot succeeds in compensating not only slow calibration drifts, but also sudden changes in its geometry. Hardware aspects of the robot-camera system are discussed.

Vector quantization for texture classification

A method for classifying and coding textures that is based upon transform vector quantization is presented. Techniques for texture classification and vector quantization similarly process small, nonoverlapping blocks of image data. Local spatial frequency features have been identified as being appropriate for texture classification, indicating that a transform vector quantization scheme should be capable of characterizing and classifying textured regions. Methods for classifying codebooks using labeled training set data are presented, producing a scheme for representation and classification of texture via vector quantization. The technique is applied to a data set consisting of seven natural textures, yielding good overall performance in comparison to other texture classification schemes. Texture classification using transform vector quantization requires little supervision, provides good classification performance and utilizes a computational structure that is suitable for implementation in practical image processing systems. >

Human visual system weighted progressive image transmission using lapped orthogonal transform/classified vector quantization

A progressive image transmission (PIT) scheme based on the classified transform vector quantization (CVQ) technique using the lapped orthogonal transform (LOT) and human visual system (HVS) weighting is proposed. Conventional block transform coding of images using the discrete cosine transform (DCT) produces, in general, undesirable blocking artifacts at low bit rates. Here image blocks are transformed using the LOT and classified into four classes based on their structural properties and further subdivided adaptively into subvectors depending on the LOT coefficient statistics with HVS weighting to improve the reconstructed image quality by adaptive bit allocation. The subvectors are vector quantized and transmitted progressively. Coding tests using computer simulations show that the LOT/CVQ-based PIT of images is an effective coding scheme. The results are also compared with those obtained using PIT/DCTVQ. The LOT/CVQ-based PIT reduces the blocking artifact significantly.

Predictive vector quantization using a neural network approach

A new predictive vector quantization (PVQ) technique capable of exploring the nonlinear dependencies in addition to the linear dependencies that exist between adjacent blocks (vectors) of pixels is introduced. The two components of the PVQ scheme, the vector predictor and the vector quantizer, are implemented by two different classes of neural networks. A multilayer perceptron is used for the predictive cornponent and Kohonen self-organizing feature maps are used to design the codebook for the vector quantizer. The multilayer perceptron uses the nonlinearity condition associated with its processing units to perform a nonlinear vector prediction. The second component of the PVQ scheme vector quantizes the residual vector that is formed by subtracting the output of the perceptron from the original input vector. The joint-optimization task of designing the two components of the PVQ scheme is also achieved. Simulation results are presented for still images with high visual quality.

Neural networks for classified vector quantization of images

Recently, the vector quantization (VQ) technique has received considerable attention and has become an effective tool for data compression. It provides high compression ratios and simple decoding processes for digital images. However, studies of practical implementation of VQ have revealed some major difficulties such as edge integrity of the reconstructed images and computational complexity of codebook design. Over the past few years, a new wave of research in neural networks has emerged. Neural network models have provided an effective means of solving computationally intensive problems. This paper proposes the implementation of classified vector quantization for image compression with neural network models. In order to preserve the edge integrity and improve the efficiency of codebook design, the proposed method includes a multilayer perceptron model for edge classification and a self-organization model for codebook design. A system architecture is proposed, and simulation results demonstrate improvements in visual quality and input encoding.

A complexity reduction technique for image vector quantization

A technique for reducing the complexity of spatial-domain image vector quantization (VQ) is proposed. The conventional spatial domain distortion measure is replaced by a transform domain subspace distortion measure. Due to the energy compaction properties of image transforms, the dimensionality of the subspace distortion measure can be reduced drastically without significantly affecting the performance of the new quantizer. A modified LBG algorithm incorporating the new distortion measure is proposed. Unlike conventional transform domain VQ, the codevector dimension is not reduced and a better image quality is guaranteed. The performance and design considerations of a real-time image encoder using the techniques are investigated. Compared with spatial domain a speed up in both codebook design time and search time is obtained for mean residual VQ, and the size of fast RAM is reduced by a factor of four. Degradation of image quality is less than 0.4 dB in PSNR.

Unrestricted multistage vector quantizers

A low-complexity, high-performance vector quantization technique for medium-to-high-bit-rate encoding based on multistage (residual) VQ is introduced. The design, a variable-rate version of multistage VQ, is the vector extension of the piecewise uniform scalar quantizers of P.F. Swaszek and J.B. Thomas (1984), and is related to the piecewise uniform VQs of F. Kuhlmann and J.A. Buckview (1988). Examples for the independent Gaussian and Laplacian sources are included. >

Dynamic finite-state vector quantisation of colour images

A dynamic finite-state vector quantisation technique for colour image compression is proposed. The proposed method efficiently exploits both the statistical redundancy between the colour components of a pixel and the high correlations between adjacent pixels. In addition, the image is compressed losslessly as compared with a colour quantised index image. Experimental results show that it can significantly reduce the storage requirement while maintaining excellent image quality.

Method of and device for speech signal coding and decoding by parameter extraction and vector quantization techniques

First Page

Multiband-excited linear predictive coder with a two-sided short-term predictor

Abstract A multiband-excited linear predictive coder with a two-sided short-term predictor is proposed. A two-sided predictor (TSP), which predicts on the basis of past and future samples within the frame, is used to reconstruct the spectral envelope. The coder is a completely frame-based, fixed bit-rate, frequency-domain coder. The excitation spectrum is split into bands whose width depends on the pitch frequency, and each of these bands is coded independently using vector quantisation techniques. Dynamic bit allocation is employed for each band based on the speech energy in that band. The synthetic spectrum is obtained as the product of the excitation spectrum and the spectral envelope. This coder combines some of the benefits of sub-band coding with those of CELP-type techniques. The coder has a highly parallel structure, and pitch periodicity is exploited effectively although the coding is independent in each frame. A 4.8 kbit/s coder was designed and tested and the performance was seen to be promising.