Context Tree Research Articles

QUC-Tree: Integrating Query Context Information for Efficient Music Retrieval

In this paper, we introduce a novel indexing scheme-query context tree (QUC-tree) to facilitate efficient query sensitive music search under different query contexts. Distinguished from the previous approaches, QUC-tree is a balanced multiway tree structure, where each level represents the data space at different dimensionality. Before the tree structure construction, principle component analysis (PCA) is applied for data analysis and transforming the raw composite features into a new feature space sorted by the importance of acoustic features. The PCA transformed data and reduced dimensions in the upper levels can alleviate suffering from dimensionality curse. To accurately mimic human perception, an extension called QUC +-tree is proposed, which further applies multivariate regression and EM based algorithm to estimate the weight of each individual feature. The comprehensive extensive experiments to evaluate the proposed structures against state-of-art techniques based on different datasets. The experimental results demonstrate the superiority of our technique.

Word predicting method, voice recognition method, and voice recognition apparatus and program using the same methods

A word predicting method for use with a voice recognition using a computer includes the steps of specifying a sentence structure of a history up to a word immediately before the word to be predicted, referring to a context tree stored in arboreal context tree storage section having information about possible structures of a sentence and a probability of appearance of words with respect to the structures at nodes, and predicting words based on the context tree and the specified sentence structure of the history.

On Finite Memory Universal Data Compression and Classification of Individual Sequences

Consider the case where consecutive blocks of letters of a semi-infinite individual sequence over a finite-alphabet are being compressed into binary sequences by some one-to-one mapping. No a priori information about is available at the encoder, which must therefore adopt a universal data-compression algorithm. It is known that if the universal Lempel-Ziv (LZ) data compression algorithm is successively applied to -blocks then the best error-free compression, for the particular individual sequence is achieved as tends to infinity. The best possible compression that may be achieved by any universal data compression algorithm for finite -blocks is discussed. It is demonstrated that context tree coding essentially achieves it. Next, consider a device called classifier (or discriminator) that observes an individual training sequence . The classifier's task is to examine individual test sequences of length and decide whether the test -sequence has the same features as those that are captured by the training sequence , or is sufficiently different, according to some appropriate criterion. Here again, it is demonstrated that a particular universal context classifier with a storage-space complexity that is linear in , is essentially optimal. This may contribute a theoretical ldquoindividual sequencerdquo justification for the Probabilistic Suffix Tree (PST) approach in learning theory and in computational biology.

Random perturbations of stochastic processes with unbounded variable length memory

We consider binary infinite order stochastic chains perturbed by a random noise. This means that at each time step, the value assumed by the chain can be randomly and independently flipped with a small fixed probability. We show that the transition probabilities of the perturbed chain are uniformly close to the corresponding transition probabilities of the original chain. As a consequence, in the case of stochastic chains with unbounded but otherwise finite variable length memory, we show that it is possible to recover the context tree of the original chain, using a suitable version of the algorithm Context, provided that the noise is small enough.

Exponential inequalities for VLMC empirical trees

A seminal paper by Rissanen, published in 1983, introduced the class of Variable Length Markov Chains and the algorithm Context which estimates the probabilistic tree generating the chain. Even if the subject was recently considered in several papers, the central question of the rate of convergence of the algorithm remained open. This is the question we address here. We provide an exponential upper bound for the probability of incorrect estimation of the probabilistic tree, as a function of the size of the sample. As a consequence we prove the almost sure consistency of the algorithm Context. We also derive exponential upper bounds for type I errors and for the probability of underestimation of the context tree. The constants appearing in the bounds are all explicit and obtained in a constructive way.

Probabilistic calling context

Calling context enhances program understanding and dynamic analyses by providing a rich representation of program location. Compared to imperative programs, object-oriented programs use more interprocedural and less intraprocedural control flow, increasing the importance of context sensitivity for analysis. However, prior online methods for computing calling context, such as stack-walking or maintaining the current location in a calling context tree, are expensive in time and space. This paper introduces a new online approach called probabilistic calling context (PCC) that continuously maintains a probabilistically unique value representing the current calling context. For millions of unique contexts, a 32-bit PCC value has few conflicts. Computing the PCC value adds 3% average overhead to a Java virtual machine. PCC is well-suited to clients that detect new or anomalous behavior since PCC values from training and production runs can be compared easily to detect new context-sensitive behavior; clients that query the PCC value at every system call, Java utility call, and Java API call add 0-9% overhead on average. PCC adds space overhead proportional to the distinct contexts stored by the client (one word per context). Our results indicate PCC is efficient and accurate enough to use in deployed software for residual testing, bug detection, and intrusion detection.

Optimal segmentation using tree models

Sequence data are abundant in application areas such as computational biology, environmental sciences, and telecommunication. Many real-life sequences have a strong segmental structure, with segments of different complexities. In this paper we study the description of sequence segments using variable length Markov chains (VLMCs), also known as tree models. We discover the segment boundaries of a sequence and at the same time we obtain a VLMC for each segment. Such a context tree contains the probability distribution vectors that capture the essential features of the corresponding segment. We use the Bayesian information criterion (BIC) and the Krichevsky-Trofimov probability (KT) to select the number of segments of a sequence. On DNA data the method selects segments that closely correspond to the annotated regions of the genes.

Universal Piecewise Linear Prediction Via Context Trees

This paper considers the problem of piecewise linear prediction from a competitive algorithm approach. In prior work, prediction algorithms have been developed that are universal with respect to the class of all linear predictors, such that they perform nearly as well, in terms of total squared prediction error, as the best linear predictor that is able to observe the entire sequence in advance. In this paper, we introduce the use of a tree, to compete against a doubly exponential number of piecewise linear (affine) models. We use the context tree to achieve the total squared prediction error performance of the best piecewise linear model that can choose both its partitioning of the regressor space and its real-valued prediction parameters within each region of the partition, based on observing the entire sequence in advance, uniformly, for every bounded individual sequence. This performance is achieved with a prediction algorithm whose complexity is only linear in the depth of the context tree per prediction. Upper bounds on the regret with respect to the best piecewise linear predictor are given for both the scalar and higher order case, and lower bounds on the regret are given for the scalar case. An explicit algorithmic description and examples demonstrating the performance of the algorithm are given.

Lossless Compression of Color Map Images by Context Tree Modeling

Significant lossless compression results of color map images have been obtained by dividing the color maps into layers and by compressing the binary layers separately using an optimized context tree model that exploits interlayer dependencies. Even though the use of a binary alphabet simplifies the context tree construction and exploits spatial dependencies efficiently, it is expected that an equivalent or better result would be obtained by operating directly on the color image without layer separation. In this paper, we extend the previous context-tree-based method to operate on color values instead of binary layers. We first generate an n-ary context tree by constructing a complete tree up to a predefined depth, and then prune out nodes that do not provide compression improvements. Experiments show that the proposed method outperforms existing methods for a large set of different color map images.

Superior Guarantees for Sequential Prediction and Lossless Compression via Alphabet Decomposition

We present worst case bounds for the learning rate of a known prediction method that is based on hierarchical applications of binary context tree weighting (CTW) predictors. A heuristic application...

Redundancy of the Context-Tree Weighting Method on Renewal and Markov Renewal Processes

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> The &lt;emphasis emphasistype="italic"&gt;Context Tree Weighting&lt;/emphasis&gt; method (CTW) is shown to be almost &lt;emphasis emphasistype="italic"&gt;adaptive&lt;/emphasis&gt; on the classes of &lt;emphasis emphasistype="italic"&gt;renewal&lt;/emphasis&gt; and &lt;emphasis emphasistype="italic"&gt;Markov renewal&lt;/emphasis&gt; processes. Up to logarithmic factor, &lt;emphasis emphasistype="smcaps"&gt;ctw&lt;/emphasis&gt; achieves the minimax pointwise redundancy described by I. Csiszár and P. Shields in &lt;emphasis emphasistype="boldital"&gt;IEEE Trans. Inf. Theory&lt;/emphasis&gt;, vol. 42, no. 6, pp. 2065–2072, Nov. 1996. This result not only complements previous results on the adaptivity of the Context-Tree Weighting method on the relatively small class of all finite context-tree sources (which encompasses the class of all finite order Markov sources), it shows that almost minimax redundancy can be achieved on massive classes of sources (classes that cannot be smoothly parameterized by subsets of finite-dimensional spaces). Moreover, it shows that (almost) adaptive compression can be achieved in a computationally efficient way on those massive classes. While previous adaptivity results for CTW could rely on the fact that any Markov source is a finite-context-tree source, this is no longer the case for renewal sources. In order to prove almost adaptivity of CTW over renewal sources, it is necessary to establish that CTW carefully balances estimation error and approximation error. </para>

Lossless compression of map contours by context tree modeling of chain codes

We consider lossless compression of digital contours in map images. The problem is attacked by the use of context-based statistical modeling and entropy coding of the chain codes. We propose to generate an optimal n-ary incomplete context tree by first constructing a complete tree up to a predefined depth and creating the optimal tree by pruning out nodes that do not provide improvement in compression. We apply this method for both vector and raster maps. Experiments show that the proposed method gives lower bit rates than the existing methods of chain codes compression for the set of test data.

Consistency of the Unlimited BIC Context Tree Estimator

The Bayesian information criterion (BIC) and Krichevsky- Trofimov (KT) version of minimum description length (MDL) principle are popular in the study of model selection. For order estimation of Markov chains, both are known to be strongly consistent when there is an upper-bound on the order. In the unbounded case, the BIC is also known to be consistent, but the KT estimator is consistent only with a bound o(log n) on the order. For context trees, a flexible generalization of Markov models widely used in data processing, the problem is more complicated both in theory and practice, given the substantially higher number of possible candidate models. Imre Csiszar and Zsolt Talata proved the consistency of BIC and KT when the hypothetical tree depths are allowed to grow as o(log n). This correspondence proves that such a restriction is not necessary for finite context sources: the BIC context tree estimator is strongly consistent even if there is no constraint at all on the size of the chosen tree. Moreover, an algorithm computing the tree minimizing the BIC criterion among all context trees in linear time is provided

Lossless image compression via bit-plane separation and multilayer context tree modeling

Color separation and highly optimized context tree mod- eling for binary layers have provided the best compression results for color map images that consist of highly complex spatial struc- tures but only a relatively few number of colors. We explore whether this kind of approach works on photographic and palette images as well. The main difficulty is that these images can have a much higher number of colors, and it is therefore much more difficult to exploit spatial dependencies via binary layers. The original contribu- tions of this work include: 1. the application of context-tree-based compression (previously designed for map images) to natural and color palette images; 2. the consideration of four different methods for bit-plane separation; and 3. Extension of the two-layer context to a multilayer context for better utilization of the crosslayer correla- tions. The proposed combination is extensively compared to state of the art lossless image compression methods. © 2006 SPIE and

Universal Divergence Estimation for Finite-Alphabet Sources

This paper studies universal estimation of divergence from the realizations of two unknown finite-alphabet sources. Two algorithms that borrow techniques from data compression are presented. The first divergence estimator applies the Burrows–Wheeler block sorting transform to the concatenation of the two realizations; consistency of this estimator is shown for all finite-memory sources. The second divergence estimator is based on the Context Tree Weighting method; consistency is shown for all sources whose memory length does not exceed a known bound. Experimental results show that both algorithms perform similarly and outperform string-matching and plug-in methods.

Accurate, efficient, and adaptive calling context profiling

Calling context profiles are used in many inter-procedural code optimizations and in overall program understanding. Unfortunately, the collection of profile information is highly intrusive due to the high frequency of method calls in most applications. Previously proposed calling-context profiling mechanisms consequently suffer from either low accuracy, high overhead, or both. We have developed a new approach for building the calling context tree at runtime, called adaptive bursting . By selectively inhibiting redundant profiling, this approach dramatically reduces overhead while preserving profile accuracy. We first demonstrate the drawbacks of previously proposed calling context profiling mechanisms. We show that a low-overhead solution using sampled stack-walking alone is less than 50% accurate, based on degree of overlap with a complete calling-context tree. We also show that a static bursting approach collects a highly accurate profile, but causes an unacceptable application slowdown. Our adaptive solution achieves 85% degree of overlap and provides an 88% hot-edge coverage when using a 0.1 hot-edge threshold, while dramatically reducing overhead compared to the static bursting approach.

Adding lossless video compression to MPEGs

In this correspondence, we propose to add a lossless compression functionality into existing MPEGs by developing a new context tree to drive arithmetic coding for lossless video compression. In comparison with the existing work on context tree design, the proposed algorithm features in 1) prefix sequence matching to locate the statistics model at the internal node nearest to the stopping point, where successful match of context sequence is broken; 2) traversing the context tree along a fixed order of context structure with a maximum number of four motion compensated errors; and 3) context thresholding to quantize the higher end of error values into a single statistics cluster. As a result,the proposed algorithm is able to achieve competitive processing speed, low computational complexity and high compression performances, which bridges the gap between universal statistics modeling and practical compression techniques. Extensive experiments show that the proposed algorithm outperforms JPEG-LS by up to 24% and CALIC by up to 22%, yet the processing time ranges from less than 2 seconds per frame to 6 seconds per frame on a typical PC computing platform.

Context tree estimation for not necessarily finite memory processes, via BIC and MDL

The concept of context tree, usually defined for finite memory processes, is extended to arbitrary stationary ergodic processes (with finite alphabet). These context trees are not necessarily complete, and may be of infinite depth. The familiar Bayesian information criterion (BIC) and minimum description length (MDL) principles are shown to provide strongly consistent estimators of the context tree, via optimization of a criterion for hypothetical context trees of finite depth, allowed to grow with the sample size n as o(logn). Algorithms are provided to compute these estimators in O(n) time, and to compute them on-line for all i les n in o(nlogn) time

MPEG-2 based lossless video compression

The authors describe an efficient algorithm design for lossless video compression by using MPEG-2 as a basic research platform. Starting from MPEG motion estimation and compensation, the proposed algorithm focuses on a context tree design to fine tune the statistics and thus optimise the estimation of conditional probabilities to drive an arithmetic coder. In comparison with the existing work on context tree design, the proposed algorithm features: (i) prefix sequence matching to locate the statistics model at the internal node nearest to the stopping point, where successful match of context sequence is broken; (ii) traversing the context tree along a fixed order of context structure with a maximum number of four motion compensated errors; and (iii) context thresholding to quantise the higher end of error values into a single statistics cluster. As a result, the proposed algorithm is able to achieve competitive processing speed, low computational complexity and high compression performances, which bridges the gap between universal statistics modelling and practical compression techniques. When JPEG-LS and CALIC, the existing state-of-the-art in lossless compression of still images, are applied to those motion compensated error-frames as well as individual non-predicted frames to formulate benchmarks, the authors' experiments illustrate that the proposed algorithm outperforms JPEG-LS by up to 24% and CALIC by up to 22%, yet the processing time ranges from less than 2 s per frame to 6 s per frame on a typical PC computing platform.

Morphological reconstruction of semantic layers in map images

Map images are composed of semantic layers depicted in arbitrary color. Color separation is often needed to divide the image into layers for storage and processing. Separation can result in severe artifacts because of the overlapping of the layers. In this work, we introduce a technique to restore the original semantic lay- ers after the color separation. The proposed restoration technique improves compression performance of the reconstructed layers in comparison to the corrupted ones when compressed by lossless algorithms such as International Communication Unit (ITU) Group 4 (TIFF G4), Portable Network Graphics (PNG), Joint Bi-level Image experts Group (JBIG), and context tree method. The resulting tech- nique also provides good visual quality of the reconstructed image layers, and can therefore be applied for selective layer removal/ extraction in other map processing applications, e.g., area