Vector Quantization Codebook Design Research Articles

On the Initialization of Swarm Intelligence Algorithms for Vector Quantization Codebook Design.

Vector Quantization (VQ) is a technique with a wide range of applications. For example, it can be used for image compression. The codebook design for VQ has great significance in the quality of the quantized signals and can benefit from the use of swarm intelligence. Initialization of the Linde-Buzo-Gray (LBG) algorithm, which is the most popular VQ codebook design algorithm, is a step that directly influences VQ performance, as the convergence speed and codebook quality depend on the initial codebook. A widely used initialization alternative is random initialization, in which the initial set of codevectors is drawn randomly from the training set. Other initialization methods can lead to a better quality of the designed codebooks. The present work evaluates the impacts of initialization strategies on swarm intelligence algorithms for codebook design in terms of the quality of the designed codebooks, assessed by the quality of the reconstructed images, and in terms of the convergence speed, evaluated by the number of iterations. Initialization strategies consist of a combination of codebooks obtained by initialization algorithms from the literature with codebooks composed of vectors randomly selected from the training set. The possibility of combining different initialization techniques provides new perspectives in the search for the quality of the VQ codebooks. Nine initialization strategies are presented, which are compared with random initialization. Initialization strategies are evaluated on the following algorithms for codebook design based on swarm clustering: modified firefly algorithm-Linde-Buzo-Gray (M-FA-LBG), modified particle swarm optimization-Linde-Buzo-Gray (M-PSO-LBG), modified fish school search-Linde-Buzo-Gray (M-FSS-LBG) and their accelerated versions (M-FA-LBGa, M-PSO-LBGa and M-FSS-LBGa) which are obtained by replacing the LBG with the accelerated LBG algorithm. The simulation results point out to the benefits of the proposed initialization strategies. The results show gains up to 4.43 dB in terms of PSNR for image Clock with M-PSO-LBG codebooks of size 512 and codebook design time savings up to 67.05% for image Clock, with M-FF-LBGa codebooks with size N=512, by using initialization strategies in substitution to Random initialization.

Modified Firefly Algorithm for Vector Quantization Codebook Design in Image Compression

In the recent days, the importance of image compression techniques is exponentially increased due to the generation of massive amount of data which needs to be stored or transmitted. Numerous approaches have been presented for effective image compression by the principle of representing images in its compact form through the avoidance of unnecessary pixels. Vector quantization (VA) is an effective method in image compression and the construction of quantization table is an important process is an important task. The compression performance and the quality of reconstructed data are based on the quantization table, which is actually a matrix of 64 integers. The quantization table selection is a complex combinatorial problem which can be resolved by the evolutionary algorithms (EA). Presently, EA became famous to resolve the real world problems in a reasonable amount of time. This chapter introduces Firefly (FF) with Teaching and learning based optimization (TLBO) algorithm termed as FF-TLBO algorithm for the selection of quantization table and introduces Firefly with Tumbling algorithm termed as FF-Tumbling algorithm for the selection of search space. As the FF algorithm faces a problem when brighter FFs are insignificant, the TLBO algorithm is integrated to it to resolve the problem and Tumbling efficiently train the algorithm to explore all direction in the solution space. This algorithm determines the best fit value for every bock as local best and best fitness value for the entire image is considered as global best. When these values are found by FF algorithm, compression process takes place by efficient image compression algorithm like Run Length Encoding and Huffman coding. The proposed FF-TLBO and FF-Tumbling algorithm is evaluated by comparing its results with existing FF algorithm using a same set of benchmark images in terms of Mean Square Error (MSE), Peak Signal to Noise Ratio (PSNR), Signal to Noise Ratio (SNR). The obtained results ensure the superior performance of FF-TLBO and FF-Tumbling algorithm over FF algorithm and make it highly useful for real time applications.

Vector quantization codebook design based on Fish School Search algorithm

Vector Quantization (VQ) has been used in image coding systems since it allows high compression rates. Codebook design can be seen as a high dimensional optimization problem and, in this scenario, swarm intelligence techniques have been used. This paper presents a new VQ codebook design algorithm based on swarm clustering. The method, based on Fish School Search (FSS) algorithm, is introduced. The FSS is embedded in Linde–Buzo–Gray (LBG) algorithm as a swarm clustering method, here called FSS-LBG. Also, a modification in the original FSS breeding operation is proposed in order to favor the exploration ability and, therefore, achieve better results in terms of PSNR of the reconstructed images. Simulation results show gains up to 1.57 dB in terms of PSNR when compared to LBG algorithm for the image Lena at 0.5625 bpp using a codebook of size 512.

VQ codebook design using modified K-means algorithm with feature classification and grouping based initialization

Vector quantization (VQ) has been successfully used in data compression and feature extraction areas. Codebook design is the essential step of VQ. The K-means algorithm is a famous data clustering technique which is also an efficient codebook design scheme. The main disadvantages of K-means algorithm lie in that the initial cluster centroids greatly affect the convergence speed and the final clustering performance. In the past two decades, many codebook initialization techniques have been proposed. However, most of these techniques do not make full use of the features of the training vectors, and some techniques require high extra computational load. This paper presents an efficient and simple technique for the conventional K-means algorithm based on feature classification and grouping. Firstly, all training vectors are classified into sixteen categories based on a two-level classifier including an edge classifier and a contrast classifier. Then, the training vectors in each category are sorted based on their norm values and divided into groups. Each group has the same size, and the centroid vector of each group is calculated as an initial codeword. Experimental results show that, compared with several typical initialization techniques, our technique can obtain a better codebook along with a faster convergence speed in a shorter time.

Genetic Simulated Annealing-Based Kernel Vector Quantization Algorithm

Genetic Algorithm (GA) has been successfully applied to codebook design for vector quantization and its candidate solutions are normally turned by LBG algorithm. In this paper, to solve premature phenomenon and falling into local optimum of GA, a new Genetic Simulated Annealing-based Kernel Vector Quantization (GSAKVQ) is proposed from a different point of view. The simulated annealing (SA) method proposed in this paper can approach the optimal solution faster than the other candidate approaches. In the frame of GA, firstly, a new special crossover operator and a mutation operator are designed for the partition-based code scheme, and then a SA operation is introduced to enlarge the exploration of the proposed algorithm, finally, the Kernel function-based fitness is introduced into GA in order to cluster those datasets with complex distribution. The proposed method has been extensively compared with other algorithms on 17 datasets clustering and four image compression problems. The experimental results show that the algorithm can achieve its superiority in terms of clustering correct rate and peak signal-to-noise ratio (PSNR), and the robustness of algorithm is also very good. In addition, we took “Lena” as an example and added Gaussian noise into the original image then adopted the proposed algorithm to compress the image with noise. Compared to the original image with noise, the reconstructed image is more distinct, and with the parameter value increasing, the value of PSNR decreases.

PSO Algorithm Applied to Codebook Design for Channel-Optimized Vector Quantization

Vector quantization (VQ) has been used in signal compression systems. However, in the scenario of image transmission, VQ is very sensitive to channel errors. An approach to decrease such sensitivity is channel-optimized vector quantization (COVQ), which involves VQ codebook design taking into account the characteristics of the channel. In the present work, particle swarm optimization (PSO) is applied to codebook design for COVQ. Simulation results are presented for a variety of bit error rates of a binary symmetric channel (BSC) and reveal the effectiveness of the method in decreasing visual impairment by blocking artifacts in the reconstructed images, overperforming conventional COVQ codebook design in terms of peak signal to noise ratio of the reconstructed images for approximately 90% of exhaustive evaluations of image transmission over BSC.

Fine granularity scalable speech coding using embedded tree-structured vector quantization

This paper proposes an efficient codebook design for tree-structured vector quantization (TSVQ) that is embedded in nature. We modify two speech coding standards by replacing their original quantizers for line spectral frequencies (LSF’s) and/or Fourier magnitudes quantization with TSVQ-based quantizers. The modified coders are fine-granular bit-rate scalable with gradual change in quality for the synthetic speech. A fast search encoding algorithm using multistage tree-structured vector quantization (MTVQ) is proposed for quantization of LSF’s. The proposed method is compared to the multipath sequential tree-assisted search (MSTS) and to the well known multipath sequential search (MSS) or M-L search algorithms.

Methods to Accelerate a Competitive Learning Algorithm Applied to VQ Codebook Desing

Abstract. Codebook design plays a crucial role in the performance of signal pro-cessing systems based on vector quantization (VQ). This paper is concerned with methods for reducing the processing time spent by a competitive learning (CL) algorithm applied to VQ codebook design. Using analytical expressions for the number of operations (multiplications, additions, subtractions and comparisons) performed by the CL algorithm, it is shown that almost all the operations are due to the nearest neighbor search (NNS). Simulation results regarding image VQ show that simple modifications introduced in CL lead to considerable number clock cyclessavings.

Unitary Space Vector Quantization Codebook Design for Precoding MIMO System

In a codebook based precoding MIMO system, the precoding codebook significantly determines the system performance. Consequently, it is crucial to design the precoding codebook, which is related to the channel fading, antenna number, spatial correlation etc. So specific channel conditions correspond to respective optimum codebooks. In this paper, in order to obtain the optimum codebooks, a universal unitary space vector quantization (USVQ) codebook design criterion is provided, which can design the optimum codebooks for various fading and spatial correlated channels with arbitrary antenna configurations. Furthermore, the unitary space K-mean (USK) algorithm is also proposed to generate the USVQ codebook, which is iterative and convergent. Simulations show that the capacities of the precoding MIMO schemes using the USVQ codebooks are very close to those of the ideal precoding cases and outperform those of the schemes using the traditional Grassmannian codebooks and the 3GPP LTE DFT (discrete Fourier transform) codebooks.

Centroid Neural Network With a Divergence Measure for GPDF Data Clustering

An unsupervised competitive neural network for efficient clustering of Gaussian probability density function (GPDF) data of continuous density hidden Markov models (CDHMMs) is proposed in this paper. The proposed unsupervised competitive neural network, called the divergence-based centroid neural network (DCNN), employs the divergence measure as its distance measure and utilizes the statistical characteristics of observation densities in the HMM for speech recognition problems. While the conventional clustering algorithms used for the vector quantization (VQ) codebook design utilize only the mean values of the observation densities in the HMM, the proposed DCNN utilizes both the mean and the covariance values. When compared with other conventional unsupervised neural networks, the DCNN successfully allocates more code vectors to the regions where GPDF data are densely distributed while it allocates fewer code vectors to the regions where GPDF data are sparsely distributed. When applied to Korean monophone recognition problems as a tool to reduce the size of the codebook, the DCNN reduced the number of GPDFs used for code vectors by 65.3% while preserving recognition accuracy. Experimental results with a divergence-based k-means algorithm and a divergence-based self-organizing map algorithm are also presented in this paper for a performance comparison.

Unitary space vector quantization codebook design for spatial correlated limited feedback MIMO system

In the transmitting, beamforming, and receiving combing (TBRC) MIMO system, a codebook based feedback strategy is usually used to provide the transmitter with the beamforming vector. The adopted codebook affects the system performance considerably. Therefore, the codebook design is a key technology in the TBRC MIMO system. In this article, the unitary space vector quantization (USVQ) codebook design criterion is proposed to design optimal codebooks for various spatial correlated MIMO channels. And the unitary space K-mean (USK) codebook generating algorithm is provided to generate the USVQ codebooks. Simulations show that the capacities of the feedback based TBRC systems using USVQ codebooks are very close to those of the ideal cases.

Parallel codebook design for vector quantization on a message passing MIMD architecture

Vector quantization (VQ) is a widely used algorithm in speech and image data compression. One of the problems of the VQ methodology is that it requires large computation time especially for large codebook size. This paper addresses two issues. The first deals with the parallel construction of the VQ codebook which can drastically reduce the training time. A master/worker parallel implementation of a VQ algorithm is proposed. The algorithm is executed on the DM-MIMD Alex AVX-2 machine using a pipeline architecture. The second issue deals with the ability of accurately predicting the machine performance. Using communication and computation models, a comparison between expected and real performance is carried out. Results show that the two models can accurately predict the performance of the machine for image data compression. Analysis of metrics normally used in parallel realization is conducted.

Vector quantization based on genetic simulated annealing

Genetic algorithm (GA) has been successfully applied to codebook design for vector quantization (VQ). However, most conventional GA-based codebook design methods need long runtime because candidate solutions must be fine tuned by LBG. In this paper, a partition-based GA is applied to codebook design, which is referred to as genetic vector quantization (GVQ). In addition, simulated annealing (SA) algorithm is also used in GVQ to get more promising results and the corresponding method is referred to as GSAVQ. Both GVQ and GSAVQ use the linear scaling technique during the calculation of objective functions and use special crossover and mutation operations in order to obtain better codebooks in much shorter CPU time. Experimental results show that both of them save more than 71–87% CPU time compared to LBG. For different codebook sizes, GVQ outperforms LBG by 1.1–2.1 dB in PSNR, and GSAVQ outperforms LBG by 1.2–2.2 dB in PSNR. In addition, GVQ and GSAVQ need a little longer CPU time than, the maximum decent (MD) algorithm, but they outperform MD by 0.2–0.5 dB in PSNR.

Three-dimensional wavelet transform video coding using symmetric codebook vector quantization

A new vector quantizer codebook design for video compression is described. This uses the notion that symmetries in the data, which are seldom captured exactly in any training dataset, are important perceptually and lead to a more robust codebook. The method is illustrated using three-dimensional (3-D) wavelet transformed video sequences.

Feature-based initialisation for image vector quantisation codebook generation

A feature-based initialisation method is developed for codebook design in vector quantisation of images. The proposed algorithm makes use of different features of the image block in training set-to-do clustering for producing the initial codebook for the LBG algorithm. It is shown that the proposed method not only speeds up the LBG process but also enhances the quality of the reproduced images.

High fidelity adaptive vector quantization at very low bit rates for progressive transmission of radiographic images

An adaptive vector quantizer (VQ) using a clustering technique known as adaptive fuzzy leader clustering (AFLC) that is similar in concept to deterministic annealing (DA) for VQ codebook design has been developed. This vector quantizer, AFLC-VQ, has been designed to vector quantize wavelet decomposed sub images with optimal bit allocation. The high-resolution sub images at each level have been statistically analyzed to conform to generalized Gaussian probability distributions by selecting the optimal number of filter taps. The adaptive characteristics of AFLC-VQ result from AFLC, an algorithm that uses self-organizing neural networks with fuzzy membership values of the input samples for upgrading the cluster centroids based on well known optimization criteria. By gen- erating codebooks containing codewords of varying bits, AFLC-VQ is capable of compressing large color/monochrome medical images at extremely low bit rates (0.1 bpp and less) and yet yielding high fidelity reconstructed images. The quality of the reconstructed im- ages formed by AFLC-VQ has been compared with JPEG and EZW, the standard and the well known wavelet based compression tech- nique (using scalar quantization), respectively, in terms of statistical performance criteria as well as visual perception. AFLC-VQ exhibits much better performance than the above techniques. JPEG and EZW were chosen as comparative benchmarks since these have been used in radiographic image compression. The superior perfor- mance of AFLC-VQ over LBG-VQ has been reported in earlier pa- pers. © 1999 SPIE and IS&T. (S1017-9909(99)01301-X)

A fast Linde-Buzo-Gray algorithm in image vector quantization

This paper presents a novel algorithm for speeding up the codebook design in image vector quantization that exploits the correlation among the pixels in an image block to compress the computational complexity of calculating the squared Euclidean distortion measures, and uses the similarity between the codevectors in the consecutive code-books during the iterative clustering-process to reduce the number of codevectors necessary to be checked for one codebook search. Verified test results have shown that the proposed algorithm can provide almost 98% reduction of the execution time when compared to the conventional Linde-Buzo-Gray (LBG) algorithm.

Fast sequential implementation of "neural-gas" network for vector quantization

Although the network proposed by Martinetz et al. in 1993 has been proven for its optimality in vector quantizer design and has been demonstrated to have good performance in time-series prediction, its high computational complexity (NlogN) makes it a slow sequential algorithm. We suggest two ideas to speedup its sequential realization: (1) using a truncated exponential function as its neighborhood function and (2) applying a new extension of the partial distance elimination method (PDE). This fast realization is compared with the original version of the neural-gas network for codebook design in image vector quantization. The comparison indicates that a speedup of five times is possible, while the quality of the resulting codebook is almost the same as that of the straightforward realization.

Codebook design for vector quantization of images based on fuzzy C-means clustering algorithm

The fuzzy C-means (FCM) algorithm has two major problems for the codebook design: tremendous memory requirement and intensive computation. This paper presents two codebook design algorithms based on FCM. The first algorithm overcomes the storage problem with the use of the codebook, rather than the membership matrix in FCM, to initiate and terminate the algorithm. A fast version is presented to further solve the computation problem. The fast algorithm partitions a training set into several classes and then the first algorithm is applied to each class separately. Experimental results show that both new algorithms achieve better picture quality. Furthermore, the fast algorithm is obvi- ously faster than LBG and FCM. © 1997 Society of Photo-Optical Instrumentation Engineers.

Codebook design in vector quantisation using a hybrid system of parallel simulated annealing and evolutionary selection

A hybrid system of parallel simulated annealing and evolutionary selection (PSAES) is introduced to codebook design in vector quantisation. It is examined for the vector space of low order DCT coefficients, and compared with local and near global optimisation techniques.