Context Tree Research Articles

Design Concerns for Integrated Scripting and Interactive Visualization in Notebook Environments.

Interactive visualization can support fluid exploration but is often limited to predetermined tasks. Scripting can support a vast range of queries but may be more cumbersome for free-form exploration. Embedding interactive visualization in scripting environments, such as computational notebooks, provides an opportunity to leverage the strengths of both direct manipulation and scripting. We investigate interactive visualization design methodology, choices, and strategies under this paradigm through a design study of calling context trees used in performance analysis, a field which exemplifies typical exploratory data analysis workflows with Big Data and hard to define problems. We first produce a formal task analysis assigning tasks to graphical or scripting contexts based on their specificity, frequency, and suitability. We then design a notebook-embedded interactive visualization and validate it with intended users. In a follow-up study, we present participants with multiple graphical and scripting interaction modes to elicit feedback about notebook-embedded visualization design, finding consensus in support of the interaction model. We report and reflect on observations regarding the process and design implications for combining visualization and scripting in notebooks.

Probabilistic prediction and context tree identification in the Goalkeeper game

In this article we address two related issues on the learning of probabilistic sequences of events. First, which features make the sequence of events generated by a stochastic chain more difficult to predict. Second, how to model the procedures employed by different learners to identify the structure of sequences of events. Playing the role of a goalkeeper in a video game, participants were told to predict step by step the successive directions—left, center or right—to which the penalty kicker would send the ball. The sequence of kicks was driven by a stochastic chain with memory of variable length. Results showed that at least three features play a role in the first issue: (1) the shape of the context tree summarizing the dependencies between present and past directions; (2) the entropy of the stochastic chain used to generate the sequences of events; (3) the existence or not of a deterministic periodic sequence underlying the sequences of events. Moreover, evidence suggests that best learners rely less on their own past choices to identify the structure of the sequences of events.

Posterior Representations for Bayesian Context Trees: Sampling, Estimation and Convergence

We revisit the Bayesian Context Trees (BCT) modelling framework for discrete time series, which was recently found to be very effective in numerous tasks including model selection, estimation and prediction. A novel representation of the induced posterior distribution on model space is derived in terms of a simple branching process, and several consequences of this are explored in theory and in practice. First, it is shown that the branching process representation leads to a simple variable-dimensional Monte Carlo sampler for the joint posterior distribution on models and parameters, which can efficiently produce independent samples. This sampler is found to be more efficient than earlier MCMC samplers for the same tasks. Then, the branching process representation is used to establish the asymptotic consistency of the BCT posterior, including the derivation of an almost-sure convergence rate. Finally, an extensive study is carried out on the performance of the induced Bayesian entropy estimator. Its utility is illustrated through both simulation experiments and real-world applications, where it is found to outperform several state-of-the-art methods.

Several Approaches for the Prediction of the Operating Modes of a Wind Turbine

Growing concern about climate change has intensified efforts to use renewable energy, with wind energy highlighted as a growing source. It is known that wind turbines are characterized by distinct operating modes that reflect production efficiency. In this work, we focus on the forecasting problem for univariate discrete-valued time series of operating modes. We define three prediction strategies to overcome the difficulties associated with missing data. These strategies are evaluated through experiments using five forecasting methods across two real-life datasets. Two of the forecasting methods have been introduced in the statistical literature as extensions of the well-known context algorithm: variable length Markov chains and Bayesian context tree. Additionally, we consider a Bayesian method based on conditional tensor factorization and two different smoothers from the classical tools for time series forecasting. After evaluating each pair prediction strategy/forecasting method in terms of prediction accuracy versus computational complexity, we provide guidance on the methods that are suitable for forecasting the time series of operating modes. The prediction results that we report demonstrate that high accuracy can be achieved with reduced computational resources.

Probabilistic Context Neighborhood model for lattices

We present the Probabilistic Context Neighborhood model designed for two-dimensional lattices as a variation of a Markov random field assuming discrete values. In this model, the neighborhood structure has a fixed geometry but a variable order, depending on the neighbors’ values. Our model extends the Probabilistic Context Tree model, originally applicable to one-dimensional space. It retains advantageous properties, such as representing the dependence neighborhood structure as a graph in a tree format, facilitating an understanding of model complexity. Furthermore, we adapt the algorithm used to estimate the Probabilistic Context Tree to estimate the parameters of the proposed model. We illustrate the accuracy of our estimation methodology through simulation studies. Additionally, we apply the Probabilistic Context Neighborhood model to spatial real-world data, showcasing its practical utility.

Context-Tree Weighting and Bayesian Context Trees: Asymptotic and Non-Asymptotic Justifications

Context-Tree Weighting and Bayesian Context Trees: Asymptotic and Non-Asymptotic Justifications

WildWood: A New Random Forest Algorithm

We introduce WildWood (WW), a new ensemble algorithm for supervised learning of Random Forest (RF) type. While standard RF algorithms use bootstrap out-of-bag samples to compute out-of-bag scores, WW uses these samples to produce improved predictions given by an aggregation of the predictions of all possible subtrees of each fully grown tree in the forest. This is achieved by aggregation with exponential weights computed over out-of-bag samples, that are computed exactly and very efficiently thanks to an algorithm called context tree weighting. This improvement, combined with a histogram strategy to accelerate split finding, makes WW fast and competitive compared with other well-established ensemble methods, such as standard RF and extreme gradient boosting algorithms.

Discovery of process variants based on trace context tree

Process variants usually exhibit a high degree of internal heterogeneity, in the sense that the executions of the process differ widely from each other due to contextual factors, human factors, or deliberate business decisions. Understanding differences among process variants helps analysts and managers to make informed decisions as to how to standardise or otherwise improve a business process. Existing process variant mining approaches typically fall short in full supporting semantic process variability mining, especially rarely taking activity behaviour relationships and trace context semantic into consideration. Here, we propose a semantic process variant discovery method, aimed at solving the difficulty of distinguishing similar-but-different behaviours directly from event logs. More specifically, we adapt concepts of benchmark logs and trace context tree to formalise context semantic of event log, to classify benchmark logs into several parts, thereby the clustered trace cohorts are mapped to discover the configurable process variants. In the experimental part, some performance metrics of the proposed method are evaluated and calculated by real-world event logs, supporting the usefulness of the proposed method. The experimental results show that the proposed method is able to distinguish similar-but-different behaviours and is superior to the characteristic trace clustering method using conventional neural networks.

Detecting renewal states in chains of variable length via intrinsic Bayes factors

Markov chains with variable length are useful parsimonious stochastic models able to generate most stationary sequence of discrete symbols. The idea is to identify the suffixes of the past, called contexts, that are relevant to predict the future symbol. Sometimes a single state is a context, and looking at the past and finding this specific state makes the further past irrelevant. States with such property are called renewal states and they can be used to split the chain into independent and identically distributed blocks. In order to identify renewal states for chains with variable length, we propose the use of Intrinsic Bayes Factor to evaluate the hypothesis that some particular state is a renewal state. In this case, the difficulty lies in integrating the marginal posterior distribution for the random context trees for general prior distribution on the space of context trees, with Dirichlet prior for the transition probabilities, and Monte Carlo methods are applied. To show the strength of our method, we analyzed artificial datasets generated from different models and one example coming from the field of Linguistics.

Perancangan Aplikasi Kompresi File Teks Menggunakan Algoritma Context Tree Weighting

Currently the use of applications as information is increasingly being used. but there is a problem that is often encountered, namely the need for a large space as a storage medium. Data compression is the process of converting a set of data into a coded form to save the need for data storage. Context Tree Weighting algorithm is a data compression algorithm. To determine the results of the compression process, compression ratio and space saving media are used. The results will be known, the compression ratio and space saving. In accordance with the results of the trials conducted, it can be seen that the data that was originally large in size can be compressed very well with the implementation of compression text files.

RETRACTED: Efficient intelligent compression and recognition system-based vision computing for computed tomography COVID images

Computed tomography (CT) image-based medical recognition is extensively used for COVID recognition as it improves recognition and scanning rate. A method for intelligent compression and recognition system-based vision computing for CT COVID (ICRS-VC-COVID) was developed. The proposed system first preprocesses lung CT COVID images. Segmentation is then used to split the image into two regions: nonregion of interest (NROI) with fractal lossy compression and region of interest with context tree weighting lossless. Subsequently, a fast discrete curvelet transform (FDCT) is applied. Finally, vector quantization is implemented through the encoder, channel, and decoder. Two experiments were conducted to test the proposed ICRS-VC-COVID. The first evaluated the segmentation compression, FDCT, wavelet transform, and discrete curvelet transform (DCT). The second evaluated the FDCT, wavelet transform, and DCT with segmentation. It demonstrates a significant improvement in performance parameters, such as mean square error, peak signal-to-noise ratio, and compression ratio. At similar computational complexity, the proposed ICRS-VC-COVID is superior to some existing techniques. Moreover, at the same bit rate, it significantly improves the quality of the image. Thus, the proposed method can enable lung CT COVID images to be applied for disease recognition with low computational power and space.

A New Context Tree Inference Algorithm for Variable Length Markov Chain Model with Applications to Biological Sequence Analyses.

The statistical inference of high-order Markov chains (MCs) for biological sequences is vital for molecular sequence analyses but can be hindered by the high dimensionality of free parameters. In the seminal article by Bühlmann and Wyner, variable length Markov chain (VLMC) model was proposed to embed the full-order MC in a sparse structured context tree. In the key procedure of tree pruning of their proposed context algorithm, the word count-based statistic for each branch was defined and compared with a fixed cutoff threshold calculated from a common chi-square distribution to prune the branch of the context tree. In this study, we find that the word counts for each branch are highly intercorrelated, resulting in non-negligible effects on the distribution of the statistic of interest. We demonstrate that the inferred context tree based on the original context algorithm by Bühlmann and Wyner, which uses a fixed cutoff threshold based on a common chi-square distribution, can be systematically biased and error prone. We denote the original context algorithm as VLMC-Biased (VLMC-B). To solve this problem, we propose a new context tree inference algorithm using an adaptive tree-pruning scheme, termed VLMC-Consistent (VLMC-C). The VLMC-C is founded on the consistent branch-specific mixed chi-square distributions calculated based on asymptotic normal distribution of multiple word patterns. We validate our theoretical branch-specific asymptotic distribution using simulated data. We compare VLMC-C with VLMC-B on context tree inference using both simulated and real genome sequence data and demonstrate that VLMC-C outperforms VLMC-B for both context tree reconstruction accuracy and model compression capacity.

Bayesian Context Trees: Modelling and Exact Inference for Discrete Time Series

AbstractWe develop a new Bayesian modelling framework for the class of higher-order, variable-memory Markov chains, and introduce an associated collection of methodological tools for exact inference with discrete time series. We show that a version of the context tree weighting alg-orithm can compute the prior predictive likelihood exa-ctly (averaged over both models and parameters), and two related algorithms are introduced, which identify the a posteriori most likely models and compute their exact posterior probabilities. All three algorithms are deterministic and have linear-time complexity. A family of variable-dimension Markov chain Monte Carlo samplers is also provided, facilitating further exploration of the posterior. The performance of the proposed methods in model selection, Markov order estimation and prediction is illustrated through simulation experiments and real-world applications with data from finance, genetics, neuroscience and animal communication. The associated algorithms are implemented in the R package BCT.

Limit theorems for chains with unbounded variable length memory which satisfy Cramer condition

We consider a class of variable length Markov chains with a binary alphabet in which context tree is defined by adding finite trees with uniformly bounded height to the vertices of an infinite comb tree. Such type of Markov chain models the spike neuron patterns and also extends the class of persistent random walks. The main interest is the limiting properties of the empirical distribution of symbols from the alphabet. We obtain the strong law of large numbers, central limit theorem, and exact asymptotics for large and moderate deviations. The presence of an intrinsic renewal structure is the subject of discussion in the literature. Proofs are based on the construction of a renewals of the chain and the applying corresponding properties of the compound (or generalized) renewal processes.

Mobile Devices Interface Adaptivity Using Ontologies

Currently, many mobile devices provide various interaction styles and modes which create complexity in the usage of interfaces. The context offers the information base for the development of Adaptive user interface (AUI) frameworks to overcome the heterogeneity. For this purpose, the ontological modeling has been made for specific context and environment. This type of philosophy states to the relationship among elements (e.g., classes, relations, or capacities etc.) with understandable satisfied representation. The context mechanisms can be examined and understood by any machine or computational framework with these formal definitions expressed in Web ontology language (WOL)/Resource description frame work (RDF). The Protégé is used to create taxonomy in which system is framed based on four contexts such as user, device, task and environment. Some competency questions and use-cases are utilized for knowledge obtaining while the information is refined through the instances of concerned parts of context tree. The consistency of the model has been verified through the reasoning software while SPARQL querying ensured the data availability in the models for defined use-cases. The semantic context model is focused to bring in the usage of adaptive environment. This exploration has finished up with a versatile, scalable and semantically verified context learning system. This model can be mapped to individual User interface (UI) display through smart calculations for versatile UIs.

AMF: Aggregated Mondrian Forests for Online Learning

AbstractRandom forest (RF) is one of the algorithms of choice in many supervised learning applications, be it classification or regression. The appeal of such tree-ensemble methods comes from a combination of several characteristics: a remarkable accuracy in a variety of tasks, a small number of parameters to tune, robustness with respect to features scaling, a reasonable computational cost for training and prediction, and their suitability in high-dimensional settings. The most commonly used RF variants, however, are ‘offline’ algorithms, which require the availability of the whole dataset at once. In this paper, we introduce AMF, an online RF algorithm based on Mondrian Forests. Using a variant of the context tree weighting algorithm, we show that it is possible to efficiently perform an exact aggregation over all prunings of the trees; in particular, this enables to obtain a truly online parameter-free algorithm which is competitive with the optimal pruning of the Mondrian tree, and thus adaptive to the unknown regularity of the regression function. Numerical experiments show that AMF is competitive with respect to several strong baselines on a large number of datasets for multi-class classification.

Retrieving the structure of probabilistic sequences of auditory stimuli from EEG data

Using a new probabilistic approach we model the relationship between sequences of auditory stimuli generated by stochastic chains and the electroencephalographic (EEG) data acquired while 19 participants were exposed to those stimuli. The structure of the chains generating the stimuli are characterized by rooted and labeled trees whose leaves, henceforth called contexts, represent the sequences of past stimuli governing the choice of the next stimulus. A classical conjecture claims that the brain assigns probabilistic models to samples of stimuli. If this is true, then the context tree generating the sequence of stimuli should be encoded in the brain activity. Using an innovative statistical procedure we show that this context tree can effectively be extracted from the EEG data, thus giving support to the classical conjecture.

Short-term responses of tree squirrels to different-sized forest patches on new clearcuts in a fragmented forest landscape

Abstract Context Tree squirrels disappear when forests are clearcut. The American red squirrel (Tamiasciurus hudsonicus) and northern flying squirrel (Glaucomys sabrinus) may occupy retention forest patches on new clearcuts, but with conflicting results with respect to area, isolation, and edge effects of patches. Aims We tested the hypothesis that abundance, reproduction and body mass of T. hudsonicus and G. sabrinus populations would be greater in large than small patches of retention forest on new clearcuts. Methods We measured demographic responses of both sciurids from 2014 to 2016 in replicated (n = 4) treatments of four sizes (ha) of retention patches (means of 0.53, 1.50, 4.13 and 18.73) in south-central British Columbia, Canada. Key results Mean abundance of T. hudsonicus per sample line was similar among treatment sites, ranging from 0.7 to 1.7 (2014), 1.0 to 1.7 (2015) and 0.5 to 1.3 (2016). There was a positive linear relationship of mean abundance of red squirrels and increased basal area of conifers in the larger patches. Mean numbers of G. sabrinus were consistently highest in the 4.13 ha patch class in all years and appeared to decline over time in the other patches with fewest captures in the smallest patch size. There was a significant positive linear relationship between mean overall abundance of G. sabrinus and total density of overstorey trees. Conclusions Mean abundance, reproduction and body mass (red squirrels) of both sciurids were generally similar among patch sizes, and hence these results did not support the hypothesis. Implications Although our results were short-term, habitat quality was apparently sufficient to maintain the occurrence of T. hudsonicus across patch sizes ranging from 0.3 to 20.0 ha. Glaucomys sabrinus also occurred in several patches (13 of 16), albeit at low and variable numbers.

A Novel Framework for Identifying Customers’ Unmet Needs on Online Social Media Using Context Tree

Customer needs and user contexts play an important role in generating ideas for new products or new functions. This study proposes a novel framework for identifying customers’ unmet needs on online social media using the Context Tree through the Hierarchical Search of Concept Spaces (HSCS) algorithm. The Context Tree represents the hierarchical structure of nodes associated with related keywords and corresponding concept spaces. Unlike other methods, the Context Tree focuses on finding the unmet needs of customers from online social media. The proposed framework is applied to extract customer needs for home appliances. Identified customer needs are used to make user scenarios, which are used to develop new functions of home appliances.

Dynamic dispatch of context-sensitive optimizations

Academia has spent much effort into making context-sensitive analyses practical, with great profit. However, the implementation of context-sensitive optimizations , in contrast to analyses, is still not practical, due to code-size explosion. This growth happens because current technology requires the cloning of full paths in the Calling Context Tree. In this paper, we present a solution to this problem. We combine finite state machines and dynamic dispatching to allow fully context-sensitive specialization while cloning only functions that are effectively optimized. This technique makes it possible to apply very liberal optimizations, such as context-sensitive constant propagation, in large programs—something that could not have been easily done before. We demonstrate the viability of our idea by formalizing it in Prolog, and implementing it in LLVM. As a proof of concept, we have used our state machines to implement context-sensitive constant propagation in LLVM. The binaries produced by traditional full cloning are 2.63 times larger than the binaries that we generate with our state machines. When applied on Mozilla Firefox, our optimization increases binary size from 7.2MB to 9.2MB. Full cloning, in contrast, yields a binary of 34MB.