Minimum Description Length Principle Research Articles

Map retrieval intention recognition based on relevance feedback and geographic semantic guidance: For better understanding user retrieval demands

Effective retrieval is essential for finding resources in demand handily amidst extensive data records in data warehouse. Mainstream map retrieval methods suffer from intention gap problem and are incapable to describe sophisticated user demands precisely due to the limits of low- and middle-level text or visual feature matching, resulting in unsatisfactory retrieval results. Such limitations are more marked when map retrieval demands were characterized with joint constraints of geographic concepts. To address this issue, we propose a map retrieval intention recognition method to perceive user demands with relevance feedback samples and geographic semantics guidance. Specifically, we construct a hierarchical intention expression model to describe retrieval goals and their multi-dimensional attribute constrains; incorporate geographic ontologies to provide semantic guidance and facilitate recognition; utilize the frequent itemset mining (FIM) algorithm Apriori to generate intention candidates from relevance feedback samples, and search for the optimal intention set by adopting the minimum description length (MDL) principle. The experiments verify the effectiveness of Apriori algorithm and MDL principle on intention recognition. The proposed method outperforms the FIM algorithm Gene Ontology (RuleGO) and the Decision Tree algorithm with Hierarchical Features (DTHF) with higher recognition accuracy and noise tolerance. Furthermore, through our sample augmentation strategy, the method yields promising recognition accuracy even when the feedback sample size is as low as ten, substantially reducing the feedback burden in human-computer interactions. We envision that the application of our method in spatial data infrastructures (SDIs), such as geoportals and catalogue services, could enhance the quality of service and user experience in geospatial data discovery.

What Are the Rules? Discovering Constraints from Data

Constraint programming and AI planning are powerful tools for solving assignment, optimization, and scheduling problems. They require, however, the rarely available combination of domain knowledge and mathematical modeling expertise. Learning constraints from exemplary solutions can close this gap and alleviate the effort of modeling. Existing approaches either require extensive user interaction, need exemplary invalid solutions that must be generated by experts at great expense, or show high noise-sensitivity. We aim to find constraints from potentially noisy solutions, without the need of user interaction. To this end, we formalize the problem in terms of the Minimum Description Length (MDL) principle, by which we select the model with the best lossless compression of the data. Solving the problem involves model counting, which is #P-hard to approximate. We therefore propose the greedy URPILS algorithm to find high-quality constraints in practice. Extensive experiments on constraint programming and AI planning benchmark data show URPILS not only finds more accurate and succinct constraints, but also is more robust to noise, and has lower sample complexity than the state of the art.

MEPSI: An MDL-Based Ensemble Pruning Approach with Structural Information

Ensemble pruning that combines a subset of individual learners generated in parallel to make predictions is an important topic in ensemble learning. Past decades have developed a lot of pruning algorithms that focus on the external behavior of learners on samples, which may lead to over-fitting. In this paper, we conjecture that the generalization performance of an ensemble is not only related to its external behavior on samples but also dependent on the internal structure of individual learners. We propose the general MEPSI approach based on Kolmogorov complexity and the Minimum Description Length (MDL) principle, which formulates the ensemble pruning task as the two-objective optimization problem that comprises the empirical error and structural information among individual learners. We also provide a concrete implementation of MEPSI on decision trees. The theoretical results provide generalization bounds for both the general MEPSI approach and tree-based implementation. The comparative experiments conducted on multiple real-world data sets demonstrate the effectiveness of our proposed method.

Discovering Sequential Patterns with Predictable Inter-event Delays

Summarizing sequential data with serial episodes allows non-trivial insight into the data generating process. Existing methods penalize gaps in pattern occurrences equally, regardless of where in the pattern these occur. This results in a strong bias against patterns with long inter-event delays, and in addition that regularity in terms of delays is not rewarded or discovered---even though both aspects provide key insight. In this paper we tackle both these problems by explicitly modeling inter-event delay distributions. That is, we are not only interested in discovering the patterns, but also in describing how many times steps typically occur between their individual events. We formalize the problem in terms of the Minimum Description Length principle, by which we say the best set of patterns is the one that compresses the data best. The resulting optimization problem does not lend itself to exact optimization, and hence we propose Hopper to heuristically mine high quality patterns. Extensive experiments show that Hopper efficiently recovers the ground truth, discovers meaningful patterns from real-world data, and outperforms existing methods in discovering long-delay patterns.

Identifying hubs in directed networks.

Nodes in networks that exhibit high connectivity, also called "hubs," play a critical role in determining the structural and functional properties of networked systems. However, there is no clear definition of what constitutes a hub node in a network, and the classification of network hubs in existing work has either been purely qualitative or relies on ad hoc criteria for thresholding continuous data that do not generalize well to networks with certain degree sequences. Here we develop a set of efficient nonparametric methods that classify hub nodes in directed networks using the Minimum Description Length principle, effectively providing a clear and principled definition for network hubs. We adapt our methods to both unweighted and weighted networks, and we demonstrate them in a range of example applications using real and synthetic network data.

Floating-point histograms for exploratory analysis of large scale real-world data sets

Histograms are among the most popular methods used in exploratory analysis to summarize univariate distributions. In particular, irregular histograms are good non-parametric density estimators that require very few parameters: the number of bins with their lengths and frequencies. Although many approaches have been proposed in the literature to infer these parameters, most existing histogram methods are difficult to exploit for exploratory analysis in the case of real-world data sets, with scalability issues, truncated data, outliers or heavy-tailed distributions. In this paper, we focus on the G-Enum histogram method, which exploits the Minimum Description Length (MDL) principle to build histograms without any user parameter. We then propose to extend this method by exploiting a new modeling space based on floating-point representation, with the objective of building histograms resistant to outliers or heavy-tailed distributions. We also suggest several heuristics and a methodology suitable for the exploratory analysis of large scale real-world data sets, whose underlying patterns are difficult to recover for digitization reasons. Extensive experiments show the benefits of the approach, evaluated with a dual objective: the accuracy of density estimation in the case of outliers or heavy-tailed distributions, and the effectiveness of the approach for exploratory data analysis.

Adaptive Multitimescale Event Detection in Nonintrusive Load Monitoring Based on Minimum Description Length Principle

Adaptive Multitimescale Event Detection in Nonintrusive Load Monitoring Based on Minimum Description Length Principle

Change-points analysis for generalized integer-valued autoregressive model via minimum description length principle

This article considers the problem of modeling a class of count time series with multiple change-points using segmented generalized integer-valued autoregressive (S-GINAR) processes. The minimum description length principle (MDL) is applied to study the statistical inference for the S-GINAR model, and the consistency results of the MDL model selection procedure are established respectively under the condition of known and unknown number of change-points. To find the “best” combination of the number of change-points, the locations of change-points, the order of each segment and its parameters, a genetic algorithm with simulated annealing is implemented to solve this difficult optimization problem. In particular, the simulated annealing process makes up for the precocious problem of the traditional genetic algorithm. Numerical results from simulation experiments and three examples of real data analysis show that the procedure has excellent empirical properties.

Combining data discretization and missing value imputation for incomplete medical datasets.

Data discretization aims to transform a set of continuous features into discrete features, thus simplifying the representation of information and making it easier to understand, use, and explain. In practice, users can take advantage of the discretization process to improve knowledge discovery and data analysis on medical domain problem datasets containing continuous features. However, certain feature values were frequently missing. Many data-mining algorithms cannot handle incomplete datasets. In this study, we considered the use of both discretization and missing-value imputation to process incomplete medical datasets, examining how the order of discretization and missing-value imputation combined influenced performance. The experimental results were obtained using seven different medical domain problem datasets: two discretizers, including the minimum description length principle (MDLP) and ChiMerge; three imputation methods, including the mean/mode, classification and regression tree (CART), and k-nearest neighbor (KNN) methods; and two classifiers, including support vector machines (SVM) and the C4.5 decision tree. The results show that a better performance can be obtained by first performing discretization followed by imputation, rather than vice versa. Furthermore, the highest classification accuracy rate was achieved by combining ChiMerge and KNN with SVM.

Use of Carotid Web Angioarchitecture for Stroke Risk Assessment

To examine the utility of carotid web (CW), carotid bifurcation, and their combined angioarchitectural measurements in assessing stroke risk. Anatomical data of the internal carotid artery (ICA), common carotid artery (CCA), and the CW was gathered as part of a retrospective study from symptomatic (stroke) and asymptomatic (non-stroke) patients with CW. We built a model of stroke risk using principal-component analysis, Firth regression trained with 5-fold cross-validation, and heuristic binary cutoffs based on Minimal Description Length principle. The study included 22 patients with a mean age of 55.9 ± 12.8 years and 72.9% being female. Eleven patients experienced an ischemic stroke. The first two principal components distinguished between stroke patients and non-stroke patients. The model revealed that ICA Pouch-Tip angle (p= 0.036), CCA Pouch-Tip angle (p= 0.036), ICA Web-Pouch angle (p= 0.036), and CCA Web-Pouch angle (p= 0.036) are the most important features associated with stroke risk. Conversely, CCA and ICA anatomy (diameter and angle) were not found to be risk factors. This pilot study demonstrates that using data from CT angiogram, carotid bifurcation, and CW angioarchitecture may be used to assess stroke risk, allowing physicians to tailor care for each patient according to risk stratification.

Description length guided nonlinear unified Granger causality analysis.

Most Granger causality analysis (GCA) methods still remain a two-stage scheme guided by different mathematical theories; both can actually be viewed as the same generalized model selection issues. Adhering to Occam's razor, we present a unified GCA (uGCA) based on the minimum description length principle. In this research, considering the common existence of nonlinearity in functional brain networks, we incorporated the nonlinear modeling procedure into the proposed uGCA method, in which an approximate representation of Taylor's expansion was adopted. Through synthetic data experiments, we revealed that nonlinear uGCA was obviously superior to its linear representation and the conventional GCA. Meanwhile, the nonlinear characteristics of high-order terms and cross-terms would be successively drowned out as noise levels increased. Then, in real fMRI data involving mental arithmetic tasks, we further illustrated that these nonlinear characteristics in fMRI data may indeed be drowned out at a high noise level, and hence a linear causal analysis procedure may be sufficient. Next, involving autism spectrum disorder patients data, compared with conventional GCA, the network property of causal connections obtained by uGCA methods appeared to be more consistent with clinical symptoms.

Transition Time Determination of Single-Molecule FRET Trajectories via Wasserstein Distance Analysis in Steady-State Variations in smFRET (WAVE).

Many biological molecules respond to external stimuli that can cause their conformational states to shift from one steady state to another. Single-molecule FRET (Fluorescence Resonance Energy Transfer) is of particular interest to not only define the steady-state conformational ensemble usually averaged out in the ensemble of molecules but also characterize the dynamics of biomolecules. To study steady-state transitions, i.e., non-equilibrium transitions, a data analysis methodology is necessary to analyze single-molecule FRET photon trajectories, which contain mixtures of contributions from two steady-state statuses and include non-equilibrium transitions. In this study, we introduce a novel methodology called WAVE (Wasserstein distance Analysis in steady-state Variations in smFRET) to detect and locate non-equilibrium transition positions in FRET trajectories. Our method first utilizes a combined STaSI-HMM (Stepwise Transitions with State Inference Hidden Markov Model) algorithm to convert the original FRET trajectories into discretized trajectories. We then apply Maximum Wasserstein Distance analysis to differentiate the FRET state compositions of the fitting trajectories before and after the non-equilibrium transition. Forward and backward algorithms, based on the Minimum Description Length (MDL) principle, are used to find the refined positions of the non-equilibrium transitions. This methodology allows us to observe changes in experimental conditions in chromophore-tagged biomolecules or vice versa.

Minimum description length clustering to measure meaningful image complexity

We present a new image complexity metric. Existing complexity metrics cannot distinguish meaningful content from noise, and give a high score to white noise images, which contain no meaningful information. We use the minimum description length principle to determine the number of clusters and designate certain points as outliers and, hence, correctly assign white noise a low score. The presented method is a step towards humans’ ability to detect when data contain a meaningful pattern. It also has similarities to theoretical ideas for measuring meaningful complexity. We conduct experiments on seven different sets of images, which show that our method assigns the most accurate scores to all images considered. Additionally, comparing the different levels of the hierarchy of clusters can reveal how complexity manifests at different scales, from local detail to global structure. We then present ablation studies showing the contribution of the components of our method, and that it continues to assign reasonable scores when the inputs are modified in certain ways, including the addition of Gaussian noise and the lowering of the resolution. Code is available at https://github.com/Lou1sM/meaningful_image_complexity.

Dimensionality selection for hyperbolic embeddings using decomposed normalized maximum likelihood code-length

Graph embedding methods are effective techniques for representing nodes and their relations in a continuous space. Specifically, the hyperbolic space is more effective than the Euclidean space for embedding graphs with tree-like structures. Thus, it is critical how to select the best dimensionality for the hyperbolic space in which a graph is embedded. This is because we cannot distinguish nodes well with dimensionality that is considerably low, whereas the embedded relations are affected by irregularities in data with excessively high dimensionality. We consider this problem from the viewpoint of statistical model selection for latent variable models. Thereafter, we propose a novel methodology for dimensionality selection based on the minimum description length principle. We aim to introduce a latent variable modeling of hyperbolic embeddings and apply the decomposed normalized maximum likelihood code-length to latent variable model selection. We empirically demonstrated the effectiveness of our method using both synthetic and real-world datasets.

Joint Direction of Arrival-Polarization Parameter Tracking Algorithm Based on Multi-Target Multi-Bernoulli Filter

This paper presents a tracking algorithm for joint estimation of direction of arrival (DOA) and polarization parameters, which exhibit dynamic behavior due to the movement of signal source carriers. The proposed algorithm addresses the challenge of real-time estimation in multi-target scenarios with an unknown number. This algorithm is built upon the Multi-target Multi-Bernoulli (MeMBer) filter algorithm, which makes use of a sensor array called Circular Orthogonal Double-Dipole (CODD). The algorithm begins by constructing a Minimum Description Length (MDL) principle, taking advantage of the characteristics of the polarization-sensitive array. This allows for adaptive estimation of the number of signal sources and facilitates the separation of the noise subspace. Subsequently, the joint parameter Multiple Signal Classification (MUSIC) spatial spectrum function is employed as the pseudo-likelihood function, overcoming the limitations imposed by unknown prior information constraints. To approximate the posterior distribution of MeMBer filters, Sequential Monte Carlo (SMC) method is utilized. The simulation results demonstrate that the proposed algorithm achieves excellent tracking accuracy in joint DOA-polarization parameter estimation, whether in scenarios with known or unknown numbers of signal sources. Moreover, the algorithm demonstrates robust tracking convergence even under low Signal-to-Noise Ratio (SNR) conditions.

Evaluating syntactic proposals using minimalist grammars and minimum description length

Many patterns found in natural language syntax have multiple pos-sible explanations or structural descriptions. Even within the cur-rently dominant Minimalist framework (Chomsky 1995, 2000), it is not uncommon to encounter multiple types of analyses for the same phenomenon proposed in the literature. A natural question, then, is whether one could evaluate and compare syntactic proposals from a quantitative point of view. In this paper, we show how an evaluation measure inspired by the minimum description length principle (Rissa-nen 1978) can be used to compare accounts of syntactic phenomena implemented as minimalist grammars (Stabler 1997), and how argu-ments for and against this kind of analysis translate into quantitative differences.

A novel approach to estimating the debt capacity of European SMEs

Research background: The concept of debt capacity assumes that a maximum value of debt ratio exists that when exceeded triggers unfavourable consequences, such as drop in market value, default or a change in the business' creditworthiness. With the current state of the art there is a priori no theoretical assurance that such a specific value exists, or rather it is represented by an interval of values. Beyond that, our understanding of debt capacity is often limited to a theoretical approximation by firm-specific factors, while the context of macroeconomic factors, especially those critical for SMEs, is neglected. Purpose of the article: The aim of this paper is to present a novel approach to estimating SMEs' debt capacity. Further, the aim is to answer the question of what firm-level and macroeconomy conditions lead to exhausting the SMEs' debt capacity and under what conditions a specific value of maximum debt capacity could be estimated. Methods: To estimate the debt capacity, we suggest a use of an information entropy minimising heuristic and the Minimal Description Length Principle. In this approach, the observed feature space is categorised into several regions. In this case, such a region represents a set of firm- and macroeconomy-specific conditions forming the debt capacity of the SMEs. To the best of our knowledge, such an approach has not yet been used in debt capacity applications. Findings & value added: We found out that the debt ratio itself provides little explanation of exhausted debt capacity, suggesting that high debt levels are compensated for by other factors. By using the suggested approach, a set of more than 100 different regions was analysed. It was found that in case of five regions (sets of conditions) the debt capacity is exhausted, as the high level of debt has significant distress consequences.

Compressing network populations with modal networks reveal structural diversity

Analyzing relational data consisting of multiple samples or layers involves critical challenges: How many networks are required to capture the variety of structures in the data? And what are the structures of these representative networks? We describe efficient nonparametric methods derived from the minimum description length principle to construct the network representations automatically. The methods input a population of networks or a multilayer network measured on a fixed set of nodes and output a small set of representative networks together with an assignment of each network sample or layer to one of the representative networks. We identify the representative networks and assign network samples to them with an efficient Monte Carlo scheme that minimizes our description length objective. For temporally ordered networks, we use a polynomial time dynamic programming approach that restricts the clusters of network layers to be temporally contiguous. These methods recover planted heterogeneity in synthetic network populations and identify essential structural heterogeneities in global trade and fossil record networks. Our methods are principled, scalable, parameter-free, and accommodate a wide range of data, providing a unified lens for exploratory analyses and preprocessing large sets of network samples.

Hierarchical Dense Pattern Detection in Tensors

Dense subtensor detection gains remarkable success in spotting anomalies and fraudulent behaviors for multi-aspect data (i.e., tensors), like in social media and event streams. Existing methods detect the densest subtensors flatly and separately, with the underlying assumption that those subtensors are exclusive. However, many real-world tensors usually present hierarchical properties, e.g., the core-periphery structure and dynamic communities in networks. It is also unexplored how to fuse the prior knowledge into dense pattern detection to capture the local behavior. In this article, we propose CatchCore , a novel framework to efficiently find the hierarchical dense subtensors. We first design a unified metric for dense subtensor detection, which can be optimized with gradient-based methods. With the proposed metric, CatchCore detects hierarchical dense subtensors through the hierarchy-wise alternative optimization and finds local dense patterns concerning some items in a query manner. Finally, we utilize the minimum description length principle to measure the quality of detection results and select the optimal hierarchical dense subtensors. Extensive experiments on synthetic and real-world datasets demonstrate that CatchCore outperforms the top competitors in accuracy for detecting dense subtensors and anomaly patterns, like network attacks. Additionally, CatchCore successfully identifies a hierarchical researcher co-authorship group with intense interactions in the DBLP dataset; it can also capture core collaboration and multi-hop relations around some query objects. Meanwhile, CatchCore also scales linearly with all aspects of tensors.

Unsupervised discretization by two-dimensional MDL-based histogram

Unsupervised discretization is a crucial step in many knowledge discovery tasks. The state-of-the-art method for one-dimensional data infers locally adaptive histograms using the minimum description length (MDL) principle, but the multi-dimensional case is far less studied: current methods consider the dimensions one at a time (if not independently), which result in discretizations based on rectangular cells of adaptive size. Unfortunately, this approach is unable to adequately characterize dependencies among dimensions and/or results in discretizations consisting of more cells (or bins) than is desirable. To address this problem, we propose an expressive model class that allows for far more flexible partitions of two-dimensional data. We extend the state of the art for the one-dimensional case to obtain a model selection problem based on the normalized maximum likelihood, a form of refined MDL. As the flexibility of our model class comes at the cost of a vast search space, we introduce a heuristic algorithm, named PALM, which partitions each dimension alternately and then merges neighboring regions, all using the MDL principle. Experiments on synthetic data show that PALM (1) accurately reveals ground truth partitions that are within the model class (i.e., the search space), given a large enough sample size; (2) approximates well a wide range of partitions outside the model class; (3) converges, in contrast to the state-of-the-art multivariate discretization method IPD. Finally, we apply our algorithm to three spatial datasets, and we demonstrate that, compared to kernel density estimation (KDE), our algorithm not only reveals more detailed density changes, but also fits unseen data better, as measured by the log-likelihood.