Tree-structured Vector Quantization Research Articles

Variable-length constrained-storage tree-structured vector quantization

Constrained storage vector quantization, (CSVQ), introduced by Chan and Gersho (1990, 1991) allows for the stagewise design of balanced tree-structured residual vector quantization codebooks with low encoding and storage complexities. On the other hand, it has been established by Makhoul et al. (1985), Riskin et al. (1991), and by Mahesh et al. (see IEEE Trans. Inform. Theory, vol.41, p.917-30, 1995) that variable-length tree-structured vector quantizer (VLTSVQ) yields better coding performance than a balanced tree-structured vector quantizer and may even outperform a full-search vector quantizer due to the nonuniform distribution of rate among the subsets of its input space. The variable-length constrained storage tree-structured vector quantization (VLCS-TSVQ) algorithm presented in this paper utilizes the codebook sharing by multiple vector sources concept as in CSVQ to greedily grow an unbalanced tree structured residual vector quantizer with constrained storage. It is demonstrated by simulations on test sets from various synthetic one dimensional (1-D) sources and real-world images that the performance of VLCS-TSVQ, whose codebook storage complexity varies linearly with rate, can come very close to the performance of greedy growth VLTSVQ of Riskin et al. and Mahesh et al. The dramatically reduced size of the overall codebook allows the transmission of the code vector probabilities as side information for source adaptive entropy coding.

Optimal bit allocation and best-basis selection for wavelet packets and TSVQ

To use wavelet packets for lossy data compression, the following issues must be addressed: quantization of the wavelet subbands, allocation of bits to each subband, and best-basis selection. We present an algorithm for wavelet packets that systematically identifies all bit allocations/best-basis selections on the lower convex hull of the rate-distortion curve. We demonstrate the algorithm on tree-structured vector quantizers used to code image subbands from the wavelet packet decomposition.

Index compressed tree-structured vector quantisation

This paper introduces a novel coding scheme based on Tree-Structured Vector Quantisation (TSVQ) scheme for image compression. The genealogical relationship among the indices of the neighbouring blocks generated from vector quantisation is exploited to improve the coding performance of TSVQ. The proposed coding scheme provides about 3.5 dB improvement over the basic TSVQ scheme and outperforms VQ schemes with memory and JPEG coding standard at low bit-rates. In addition its performance is comparable with address VQ but with much less complexity.

A strategy for satellite data archival. Low noise variable-rate vector quantization with application to AVHRR satellite images: A tutorial review

Abstract The volume of satellite data amassed by modern day weather and climate satellites is so enormous that it has become virtually impossible for researchers to access the original resolution data collected by the satellites. Typically, researchers are forced to deal with lower resolution reduced data (e.g. cloud cover or temperatures) and often at a resolution degraded by one to three orders of magnitude when compared with the resolution of the original data. The uncertainties in the reduced data are often unknown. This state of affairs will only get worse, since the data to be collected under the guidance of NASA’s global change program may increase by several orders of magnitude in the coming decades. We need to experiment with mathematically rigorous ways to tame the original data without significantly degrading the information content. Compression of original data by objective mathematical techniques is a promising approach. This study adopts recent compression techniques developed in the field of communications and applies them to weather satellite data. Typically, these techniques compress the raw data by factors ranging from 10 to 100. The compressed data can be decompressed to retrieve the near original data at the site of the user. The mean error in the compression–decompression process varies from a few percent to several percent. As a demonstration, we consider the advanced very high resolution radiometer (AVHRR) radiances with a nadir resolution of 1 km ×1 km. For this demonstration, we adopt a well understood compression technique which is the so-called vector quantization technique. Several vector quantization techniques (full-search, tree-search, pruned-balanced-tree, greedy-tree and pruned-greedy-tree) are compared in performance and ease of implementation. The discussion focuses on the pruned greedy tree-structured vector quantizer because it is highly suited to the compression of AVHRR satellite images. For the case considered here, visual and scientific reproducibility of the original high resolution images are very good. The rms error for roughly 95% of the pixels in a scene is within 2%, even at a 32 : 1 compression ratio. The error in spatially averaged fields is less than 0.1% for averaging scales in excess of 50 km ×50 km. Some important spatial structural information is lost, however. It is found that the same image when compressed using JPEG standard shows significant loss of numerical accuracy at the same compression ratio of 32 : 1. But improvements and developments in compression techniques can minimize these errors and afford researchers the luxury of storing and working with high resolution data at roughly at about 0.03 of the space required by the original data.

Deadzone-constrained pruned tree-structured vector quantization

This article proposes deadzone-constrained pruned tree-structured vector quantization (DCPTSVQ) which is suitable for transform coding of image or speech. DCPTSVQ applied to transformed signals considerably improves the rate-distortion performance with remarkably reduced codebook storage when compared with conventional PTSVQ.

Constrained-storage vector quantization with a universal codebook

Many image compression techniques require the quantization of multiple vector sources with significantly different distributions. With vector quantization (VQ), these sources are optimally quantized using separate codebooks, which may collectively require an enormous memory space. Since storage is limited in most applications, a convenient way to gracefully trade between performance and storage is needed. Earlier work addressed this problem by clustering the multiple sources into a small number of source groups, where each group shares a codebook. We propose a new solution based on a size-limited universal codebook that can be viewed as the union of overlapping source codebooks. This framework allows each source codebook to consist of any desired subset of the universal code vectors and provides greater design flexibility which improves the storage-constrained performance. A key feature of this approach is that no two sources need be encoded at the same rate. An additional advantage of the proposed method is its close relation to universal, adaptive, finite-state and classified quantization. Necessary conditions for optimality of the universal codebook and the extracted source codebooks are derived. An iterative design algorithm is introduced to obtain a solution satisfying these conditions. Possible applications of the proposed technique are enumerated, and its effectiveness is illustrated for coding of images using finite-state vector quantization, multistage vector quantization, and tree-structured vector quantization.

Reduced storage VQ via secondary quantization

This paper introduces methods for reducing the table storage required for encoding and decoding with unstructured vector quantization (UVQ) or tree-structured vector quantization (TSVQ). Specifically, a low-storage secondary quantizer is used to compress the code vectors (and test vectors) of the primary quantizer. The relative advantages of uniform and nonuniform secondary quantization are investigated. A Linde-Buzo-Gray (LBG) like algorithm that optimizes the primary UVQ codebook for a given secondary codebook and another that jointly optimizes both primary and secondary codebooks are presented. In comparison to conventional methods, it is found that significant storage reduction is possible (typically a factor of two to three) with little loss of signal-to-noise ratio (SNR). Moreover, when reducing dimension is considered as another method of reducing storage, it is found that the best strategy is a combination of both. The method of secondary quantization is also applied to TSVQ to reduce the table storage required for both encoding and decoding. It is shown that by exploiting the correlation among the test vectors in the tree, both encoder and decoder storage can be significantly reduced with little loss of SNR--by a factor of about four (or two) relative to the conventional method of storing test vectors (or test hyperplanes).

Medical image compression and vector quantization

In this paper, we describe a particular set of algorithms for clustering and show how they lead to codes which can be used to com- press images. The approach is called tree-structured vector quantization (TSVQ) and amounts to a binary tree-structured two-means clustering, very much in the spirit of CART. This coding is thereafter put into the larger framework of information theory. Finally, we report the method- ology for how image compression was applied in a clinical setting, where the medical issue was the measurement of major blood vessels in the chest and the technology was magnetic resonance (MR) imaging. Mea- suring the sizes of blood vessels, of other organs and of tumors is fun- damental to evaluating aneurysms, especially prior to surgery. We argue for digital approaches to imaging in general, two benefits being improved archiving and transmission, and another improved clinical usefulness through the application of digital image processing. These goals seem particularly appropriate for technologies like MR that are inherently digital. However, even in this modern age, archiving the images of a busy radiological service is not possible without substantially compress- ing them. This means that the codes by which images are stored digi- tally, whether they arise from TSVQ or not, need to be lossy, that is, not invertible. Since lossy coding necessarily entails the loss of digital in- formation, it behooves those who recommend it to demonstrate that the quality of medicine practiced is not diminished thereby. There is a grow- ing literature concerning the impact of lossy compression upon tasks that involve detection. However, we are not aware of similar studies of mea- surement. We feel that the study reported here of 30 scans compressed to 5 different levels, with measurements being made by 3 accomplished radiologists, is consistent with 16:1 lossy compression as we practice it being acceptable for the problem at hand.

Fast encoding algorithms for tree-structured vector quantization

To overcome extensive computations of searching codewords in the encoding process of vector quantization (VQ), several fast coding approaches has been proposed. As far as we know, the tree-structured search method is the fastest. In this paper, we further develop the encoding algorithm with fast search (EAWFS) and encoding algorithm with fast comparison (EAWFC) to speed up the encoding process of tree-structured vector quantization (TSVQ). These two approaches are also applicable to pruned tree-structured vector quantization (PTSVQ). The computational complexity of EAWFC is about half of that of the available method for TSVQ. For several encoded real images, EAWFS and EAWFC have a shorter execution time than that of the available algorithm. For images with a few edges, EAWFS will outperform EAWFC. EAWFS may theoretically obtain a lower distortion than that of the conventional method or EAWFC. However from our experiments, we have found that these three methods have the same distortion.

Recursive partitioning to reduce distortion

Adaptive partitioning of a multidimensional feature space plays a fundamental role in the design of data-compression schemes. Most partition-based design methods operate in an iterative fashion, seeking to reduce distortion at each stage of their operation by implementing a linear split of a selected cell. The operation and eventual outcome of such methods is easily described in terms of binary tree-structured vector quantizers. This paper considers a class of simple growing procedures for tree-structured vector quantizers. Of primary interest is the asymptotic distortion of quantizers produced by the unsupervised implementation of the procedures. It is shown that application of the procedures to a convergent sequence of distributions with a suitable limit yields quantizers whose distortion tends to zero. Analogous results are established for tree-structured vector quantizers produced from stationary ergodic training data. The analysis is applicable to procedures employing both axis-parallel and oblique splitting, and a variety of distortion measures. The results of the paper apply directly to unsupervised procedures that may be efficiently implemented on a digital computer.

New tree-structured vector quantization with closest-coupled multipath searching method

We propose a new method to improve the image quality and the encoding time of tree-structured vector quantization (TSVQ). We call this new method the closest-coupled tree-structured vector quantization (CCTSVQ). In this method, we use the closest-neighboring pointers to enlarge the search range of each search path and guide all search paths into more appropriate codewords. CCTSVQ therefore improves the image quality of TSVQ. This fact is shown in our experimental results. Furthermore, in these experimental results, we see that the encoding time of CCTSVQ is always faster than that of TSVQ, based on the same image quality.

Nonparametric classifier design using greedy tree-structured vector quantization technique

In this paper, we propose a novel tree-structured vector quantization (TSVQ) design algorithm for the applications of nonparametric pattern recognition. The TSVQ design algorithm is used to reduce the large size of the design sets required by a nonparametric classifier. For an N-class problem, the TSVQ consists of N branches with one for each class. Using the design sets as training data, the algorithm splits the leaf nodes in a greedy manner to minimize the classification error rate for tree-growing. Simulation results show that the classifiers designed using this new algorithm require less classification time than that required by other design set reduction algorithms. In addition, in many cases, the new classifiers enjoy almost the same low error rate as that of traditional k-NN nonparametric classifiers.

Conditional arithmetic coding of tree structured vector quantisation indices with reduced memory

The authors propose an efficient conditional arithmetic coding scheme for tree structured vector quantisation (TSVQ) systems which effectively exploits interblock correlation. A memory reduction method for the conditional arithmetic coding scheme is also presented. Simulation results show that the proposed scheme provides remarkable bit-rate reduction performance.

Pruned tree-structured vector quantisation with dead zones

The authors present pruned tree-structured vector quantisation with dead zones (PTSVQWDZ) which is suitable for transforming image or speech coding. PTSVQWDZ considerably improves the rate-distortion performance with a significantly reduced codebook storage when compared with the conventional PTSVQ.

Vector quantization with compressed codebooks

Abstract Codebook storage is the key limitation in reducing distortion of vector quantization (VQ). We propose an effective lossless compression scheme with reduced computational requirements. Variable precision representation (VPR) for each vector y stores exp( y ), the number of leading bits which are zero in all elements, and avoids storing the zero bits. We show that storing the difference of centroid codevectors effectively removes the redundancy in the codebook. This difference is non-stationary. As the mean square error of the VQ encoder decreases, the added codevector differences become smaller and more amenable to compression. In mean-residual VQ, VPR can save more than 50% in storage, and achieve more than 75% bit reduction of entropy coding. Moreover, VPR reduces the cost of VQ. The VPR inner product module has 10% less area × time cost than the conventional inner product. The designs are verified and their speed estimated for 1.0-μ CMOS gate array technology. Using tree-structured VQ, they can encode in real-time television quality video (720 × 576 pixels × 30 frames/s).

Multistage storage- and entropy-constrained tree-structured vector quantization

This paper proposes a new algorithm for the design of a tree-structured vector quantizer that operates under storage and entropy constraints; hence, it is called the storage- and entropy-constrained tree-structured vector quantizer (SECTSVQ). The algorithm uses the tree-growing approach and designs the tree one stage/layer at a time. The constraints on the rate and storage size, i.e., the number of nodes or codewords, at each stage are specified prior to the design procedure. While growing the tree, at each stage the algorithm optimally allocates the rate and the number of nodes available for the current stage to the nodes of the previous stage using the dynamic programming technique. The nodes of the current stage are then determined based on the allocations. In addition to being useful as a tree-structured VQ, SECTSVQ is particularly suited for application in progressive transmission. Moreover, the optimal allocation technique used in the design can be effectively applied to other optimal resource allocation problems. The SECTSVQ algorithm is implemented for various sources and is shown to compare favorably with other VQs.

Vanishing distortion and shrinking cells

We establish an asymptotic connection between vanishing rth-power distortion and shrinking cell diameters for vector quantizers with convex cells. As a consequence, a number of shrinking cell conditions may be easily verified by showing that the quantizers in question have distortion tending to zero. This also plays an important role in the asymptotic analysis of a common greedy growing scheme for tree-structured vector quantizers.

Termination and continuity of greedy growing for tree-structured vector quantizers

Tree-structured vector quantizers (TSVQ) provide a computationally efficient, variable-rate method of compressing vector-valued data. In applications, the problem of designing a TSVQ from empirical training data is critical. Greedy growing algorithms are a common and effective approach to the design problem. They are recursive procedures that produce a TSVQ one node at a time by optimizing a simple splitting criterion at each step. While unsupervised greedy growing algorithms are well-understood from an experimental point of view, there has been little theory to support their use, or to examine their behavior on large training sets. The authors present a rigorous analysis of a greedy growing algorithm proposed by Riskin (1990), Riskin and Gray (1991), and Balakrishnan (1991). The first part of the paper is a description of the algorithm and an examination of its asymptotic behavior as it applies to a fixed, absolutely continuous distribution. The second part of the paper establishes the structural consistency of the algorithm with respect to a convergent sequence of distributions. As an application, the authors obtain results concerning the large-sample empirical behavior of the algorithm when it is applied to stationary ergodic training vectors.

Entropy-constrained tree-structured vector quantizer design

Current methods for the design of pruned or unbalanced tree-structured vector quantizers such as the generalized Breiman-Friedman-Olshen-Stone (GBFOS) algorithm proposed in 1980 are effective, but suffer from several shortcomings. We identify and clarify issues of suboptimality including greedy growing, the suboptimal encoding rule, and the need for time sharing between quantizers to achieve arbitrary rates. We then present the leaf-optimal tree design (LOTD) method which, with a modest increase in design complexity, alters and reoptimizes tree structures obtained from conventional procedures. There are two main advantages over existing methods. First, the optimal entropy-constrained nearest-neighbor rule is used for encoding at the leaves; second, explicit quantizer solutions are obtained at all rates without recourse to time sharing. We show that performance improvement is theoretically guaranteed. Simulation results for image coding demonstrate that close to 1 dB reduction of distortion for a given rate can be achieved by this technique relative to the GBFOS method.

Two fast nearest neighbor searching algorithms for image vector quantization

In this paper, two efficient codebook searching algorithms for vector quantization (VQ) are presented. The first fast search algorithm utilizes the compactness property of signal energy on transform domain and the geometrical relations between the input vector and every codevector to eliminate those codevectors that have no chance to be the closest codeword of the input vector. It achieves a full search equivalent performance. As compared with other fast methods of the same kind, this algorithm requires the fewest multiplications and the least total times of distortion measurements. Then, a suboptimal searching method, which sacrifices the reconstructed signal quality to speed up the search of nearest neighbor, is presented. This algorithm performs the search process on predefined small subcodebooks instead of the whole codebook for the closest codevector. Experimental results show that this method not only needs less CPU time to encode an image but also encounters less loss of reconstructed signal quality than tree-structured VQ does.