Kolmogorov Complexity Research Articles

Research on public satisfaction of government typhoon emergency management under artificial intelligence: An empirical analysis based on Xuwen County

<abstract> <p>Typhoon natural disasters belong to one of the four major categories of public safety events. Typhoons have stochastic uncertainty and dynamic complexity, and frequent typhoons often cause heavy casualties and property losses in China's coastal areas, seriously affecting economic development and social stability. With the rapid development of artificial intelligence (AI) technology, intelligent disaster prevention and mitigation will become the trend of future development and a hot spot for research. Based on reviewing the current situation and trend of development, this study compares and analyzes the public satisfaction of communities using traditional technology methods and AI technology applications in typhoon disaster emergency management by constructing a public satisfaction model through the literature review, taking Xuwen County, China, as an example. The study shows that AI technology has an important role in the 3 main aspects of early identification, risk assessment, risk prevention and control, and provides a new technical approach to typhoon disaster emergency management. Finally, we propose the construction scheme of the typhoon emergency management system based on AI.</p> </abstract>

Exponential ergodicity for a stochastic two-layer quasi-geostrophic model

Ergodic properties of a stochastic medium complexity model for atmosphere and ocean dynamics are analyzed. More specifically, a two-layer quasi-geostrophic model for geophysical flows is studied, with the upper layer being perturbed by additive noise. This model is popular in the geosciences, for instance to study the effects of a stochastic wind forcing on the ocean. A rigorous mathematical analysis however meets with the challenge that in the model under study, the noise configuration is spatially degenerate as the stochastic forcing acts only on the top layer. Exponential convergence of solutions laws to the invariant measure is established, implying a spectral gap of the associated Markov semigroup on a space of Hölder continuous functions. The approach provides a general framework for generalized coupling techniques suitable for applications to dissipative SPDEs. In case of the two-layer quasi-geostrophic model, the results require the second layer to obey a certain passivity condition.

Inequalities for space-bounded Kolmogorov complexity

Finding all linear inequalities for entropies remains an important open question in information theory. For a long time the only known inequalities for entropies of tuples of random variables were Shannon (submodularity) inequalities. Only in 1998 Zhang and Yeung 1998 found the first inequality that cannot be represented as a convex combination of Shannon inequalities, and several other non-Shannon inequalities were found soon after that. It turned out that the class of linear inequalities for entropies is rather fundamental, since the same class can be equivalently defined in terms of subgroup sizes or projections of multidimensional sets (Chan 2001, Chan, Yeung 2002, Romashchenko, Shen, Vereshchagin 2000). The non-Shannon inequalities have interesting applications (e.g., to proofs of lower bounds for the information ratio of secret sharing schemes). Still the class of linear inequalities for entropies is not well understood, though some partial results are known (e.g., Matúš has shown in 2007 that this class cannot be generated by a finite family of inequalities). This class also appears in algorithmic information theory: the same linear inequalities are true for Shannon entropies of tuples of random variables and Kolmogorov complexities of tuples of strings (Hammer et al., 1997). This parallelism started with the Kolmogorov–Levin formula 1968 for the complexity of pairs of strings with logarithmic precision. Longpré proved in 1986 a version of this formula for the space-bounded complexities. In this paper we prove a stronger version of Longpré’s result with a tighter space bound, using Sipser’s trick 1980. Then, using this result, we show that every linear inequality that is true for complexities or entropies, is also true for space-bounded Kolmogorov complexities with a polynomial space overhead, thus extending the parallelism to the space-bounded algorithmic information theory.

Quantifiers satisfying semantic universals have shorter minimal description length

Despite wide variation among natural languages, there are linguistic properties thought to be universal to all or nearly all languages. Here, we consider universals at the semantic level, in the domain of quantifiers, which are given by the properties of monotonicity, quantity, and conservativity, and we investigate whether these universals might be explained by differences in complexity. First, we use a minimal pair methodology and compare the complexities of individual quantifiers using approximate Kolmogorov complexity. Second, we use a simple yet expressive grammar to generate a large collection of quantifiers and we investigate their complexities at an aggregate level in terms of both their minimal description lengths and their approximate Kolmogorov complexities. For minimal description length we find that quantifiers satisfying semantic universals are simpler: they have a shorter minimal description length. For approximate Kolmogorov complexity we find that monotone quantifiers have a lower Kolmogorov complexity than non-monotone quantifiers and for quantity and conservativity we find that approximate Kolmogorov complexity does not scale robustly. These results suggest that the simplicity of quantifier meanings, in terms of their minimal description length, partially explains the presence of semantic universals in the domain of quantifiers.

Predicting phenotype transition probabilities via conditional algorithmic probability approximations.

Unravelling the structure of genotype-phenotype (GP) maps is an important problem in biology. Recently, arguments inspired by algorithmic information theory (AIT) and Kolmogorov complexity have been invoked to uncover simplicity bias in GP maps, an exponentially decaying upper bound in phenotype probability with the increasing phenotype descriptional complexity. This means that phenotypes with many genotypes assigned via the GP map must be simple, while complex phenotypes must have few genotypes assigned. Here, we use similar arguments to bound the probability P(x → y) that phenotype x, upon random genetic mutation, transitions to phenotype y. The bound is [Formula: see text], where [Formula: see text] is the estimated conditional complexity of y given x, quantifying how much extra information is required to make y given access to x. This upper bound is related to the conditional form of algorithmic probability from AIT. We demonstrate the practical applicability of our derived bound by predicting phenotype transition probabilities (and other related quantities) in simulations of RNA and protein secondary structures. Our work contributes to a general mathematical understanding of GP maps and may facilitate the prediction of transition probabilities directly from examining phenotype themselves, without utilizing detailed knowledge of the GP map.

A note on a priori forecasting and simplicity bias in time series

To what extent can we forecast a time series without fitting to historical data? Can universal patterns of probability help in this task? Deep relations between pattern Kolmogorov complexity and pattern probability have recently been used to make a priori probability predictions in a variety of systems in physics, biology and engineering. Here we study simplicity bias (SB) – an exponential upper bound decay in pattern probability with increasing complexity – in discretised time series extracted from the World Bank Open Data collection. We predict upper bounds on the probability of discretised series patterns, without fitting to trends in the data. Thus we perform a kind of ‘forecasting without training data’, predicting time series shape patterns a priori, but not the actual numerical value of the series. Additionally we make predictions about which of two discretised series is more likely with accuracy of ∼80%, much higher than a 50% baseline rate, just by using the complexity of each series. These results point to a promising perspective on practical time series forecasting and integration with machine learning methods.

PAC-learning gains of Turing machines over circuits and neural networks

A caveat to many applications of the current Deep Learning approach is the need for large-scale data. One improvement suggested by Kolmogorov Complexity results is to apply the minimum description length principle with computationally universal models. We study the potential gains in sample efficiency that this approach can bring in principle. We use polynomial-time Turing machines to represent computationally universal models and Boolean circuits to represent Artificial Neural Networks (ANNs) acting on finite-precision digits.Our analysis unravels direct links between our question and Computational Complexity results. We provide lower and upper bounds on the potential gains in sample efficiency between the MDL applied with Turing machines instead of ANNs. Our bounds depend on the bit-size of the input of the Boolean function to be learned. Furthermore, we highlight close relationships between classical open problems in Circuit Complexity and the tightness of these bounds.

Hodology

A hodological law causes the evolution of the universe to tend to follow particular types of path. I give simple illustrations in toy models and discuss how Kolmogorov complexity characterises the extent to which hodological laws explain, rather than merely describe, data.

Cryptographic hardness under projections for time-bounded Kolmogorov complexity

A version of time-bounded Kolmogorov complexity, denoted KT, has received attention in the past several years, due to its close connection to circuit complexity and to the Minimum Circuit Size Problem MCSP. Essentially all results about the complexity of MCSP hold also for MKTP (the problem of computing the KT complexity of a string). Both MKTP and MCSP are hard for SZK (Statistical Zero Knowledge) under BPP-Turing reductions; neither is known to be NP-complete.Recently, some hardness results for MKTP were proved that are not (yet) known to hold for MCSP. In particular, MKTP is hard for DET (a subclass of P) under nonuniform ≤mNC0 reductions. In this paper, we improve this, to show that MKTP‾ is hard for the (apparently larger) class NISZKL under not only ≤mNC0 reductions but even under projections. Also MKTP‾ is hard for NISZK under ≤mP/poly reductions. Here, NISZK is the class of problems with non-interactive zero-knowledge proofs, and NISZKL is the non-interactive version of the class SZKL that was studied by Dvir et al.As an application, we provide several improved worst-case to average-case reductions to problems in NP, and we obtain a new lower bound on MKTP (which is currently not known to hold for MCSP).

Dietary Phenolic Compounds as Anticancer Natural Drugs: Recent Update on Molecular Mechanisms and Clinical Trials.

Given the stochastic complexity of cancer diseases, the development of chemotherapeutic drugs is almost limited by problems of selectivity and side effects. Furthermore, an increasing number of protective approaches have been recently considered as the main way to limit these pathologies. Natural bioactive compounds, and particularly dietary phenolic compounds, showed major protective and therapeutic effects against different types of human cancers. Indeed, phenolic substances have functional groups that allow them to exert several anti-cancer mechanisms, such as the induction of apoptosis, autophagy, cell cycle arrest at different stages, and the inhibition of telomerase. In addition, in vivo studies show that these phenolic compounds also have anti-angiogenic effects via the inhibition of invasion and angiogenesis. Moreover, clinical studies have already highlighted certain phenolic compounds producing clinical effects alone, or in combination with drugs used in chemotherapy. In the present work, we present a major advance in research concerning the mechanisms of action of the different phenolic compounds that are contained in food medicinal plants, as well as evidence from the clinical trials that focus on them.

Information disclosure in the framework of Kolmogorov complexity

We consider the network consisting of three nodes 1, 2, 3 connected by two open channels 1→2 and 1→3. The information present in the node 1 consists of four strings x,y,z,w. The nodes 2, 3 know x,w and need to know y,z, respectively. We want to arrange transmission of information over the channels so that both nodes 2 and 3 learn what they need and the disclosure of information is as small as possible. By information disclosure we mean the amount of information in the strings transmitted through channels about x,y,z,w (or about x,w). We are also interested in whether it is possible to minimize the disclosure of information and simultaneously minimize the length of words transferred through the channels.

A comparison of microbial composition under three tree ecosystems using the stochastic process and network complexity approaches.

Soil microbes act as “players” in regulating biogeochemical cycles, whereas environmental heterogeneity drives microbial community assembly patterns and is influenced by stochastic and deterministic ecological processes. Currently, the limited understanding of soil microbial community assembly patterns and interactions under temperate forest stand differences pose a challenge in studying the soil microbial involvement during the succession from coniferous to broad-leaved forests. This study investigated the changes in soil bacterial and fungal community diversity and community structure at the regional scale and identified the pathways influencing soil microbial assembly patterns and their interactions. The results showed that broad-leaved forest cover in temperate forests significantly increased soil pH, and effectively increased soil water content, total carbon (TC), total nitrogen (TN), and total phosphorus (TP) contents. Both soil bacterial and fungal alpha diversity indices were correlated with soil physicochemical properties, especially in broad-leaved forest. The bacterial and fungal community composition of coniferous forest was dominated by deterministic process (bacteria: 69.4%; fungi: 88.9%), while the bacterial community composition of broad-leaved forest was dominated by stochastic process (77.8%) and the fungal community composition was dominated by deterministic process (52.8%). Proteobacteria, Acidobacteriota, Actinobacteriota, and Verrucomicrobiota were the dominant phyla of soil bacterial communities in temperate forests. Whereas Ascomycota, Mortierellomycota, Basidiomycota, and Rozellomycota were the dominant phyla of soil fungal communities in temperate forests. Most members of dominant phylum were regulated by soil physical and chemical properties. In addition, the succession from temperate coniferous forest to broad-leaved forest was conducive to maintaining the complex network of soil bacteria and fungi, and the top 20 degree of the major taxa in the network reflected the positive response of microbial interactions to the changes of soil nutrients during forest succession. This study not only shows the mechanism by which species differences in temperate forests of northern China affect soil microbial community assembly processes, but also further emphasizes the importance of the soil microbiome as a key ecosystem factor through co-occurrence network analysis.

Kolmogorov complexity metrics in assessing L2 proficiency: An information-theoretic approach.

Based on 774 argumentative writings produced by Chinese English as a foreign language (EFL) learners, this study examined the extent to which Kolmogorov complexity metrics can distinguish the proficiency levels of beginner, lower-intermediate, and upper-intermediate second language (L2) English learners. Kolmogorov complexity metric is a holistic information-theoretic approach, which measures three facets of linguistic complexity, i.e., overall, syntactic, and morphological complexity simultaneously. To assess its validity in distinguishing L2 proficiency, Kolmogorov complexity metric is compared with traditional syntactic and morphological complexity metrics as well as fine-grained syntactic complexity metrics. Results showed that Kolmogorov overall and syntactic complexity could significantly distinguish any adjacent pair of L2 levels, serving as the best separators explored in the present study. Neither Kolmogorov morphological complexity nor other complexity metrics at both the syntactic and morphological levels can distinguish between all pairs of adjacent levels. Results of correlation analysis showed that Kolmogorov syntactic complexity was not or weakly correlated with all the fine-grained syntactic complexity metrics, indicating that they may address distinct linguistic features and can complement each other to better predict different proficiency levels.

A Hybrid Stochastic-Deterministic Minibatch Proximal Gradient Method for Efficient Optimization and Generalization.

Despite the success of stochastic variance-reduced gradient (SVRG) algorithms in solving large-scale problems, their stochastic gradient complexity often scales linearly with data size and is expensive for huge data. Accordingly, we propose a hybrid stochastic-deterministic minibatch proximal gradient~(HSDMPG) algorithm for strongly convex problems with linear prediction structure, e.g.~least squares and logistic/softmax regression. HSDMPG~enjoys improved computational complexity that is data-size-independent for large-scale problems. It iteratively samples an evolving~minibatch of individual losses to estimate the original problem, and efficiently minimizes the sampled smaller-sized subproblems. For strongly convex loss of n components, HSDMPG~attains an ϵ-optimization-error within [Formula: see text] stochastic gradient evaluations, where κ is condition number, ζ = 1 for quadratic loss and ζ = 2 for generic loss. For large-scale problems, our complexity outperforms those of SVRG-type algorithms with/without dependence on data size. Particularly, when ϵ = O(1/√n) which matches the intrinsic excess error of a learning model and is sufficient for generalization, our complexity for quadratic and generic losses is respectively O (n0.5log2(n)) and O (n0.5log3(n)), which for the first time achieves optimal generalization in less than a single pass over data. Besides, we extend HSDMPG~to online strongly convex problems and prove its higher efficiency over the prior algorithms. Numerical results demonstrate the computational advantages of~HSDMPG.

Second quantised information distance

The Kolmogorov complexity of a string is the minimum length of a programme that can produce that string. Information distance between two strings based on Kolmogorov complexity is defined as the minimum length of a programme that can transform either string into the other one, both ways. The second quantised Kolmogorov complexity of a quantum state is the minimum average length of a quantum programme that can reproduce that state. In this paper, a second quantised information distance is defined based on the second quantised Kolmogorov complexity. It is described as the minimum average length of a transformation quantum programme between two quantum states. This distance's basic properties are discussed. A practical analogue of quantum information distance is also developed based on quantum data compression.

Model Complexity in Statistical Manifolds: The Role of Curvature

Model complexity plays an essential role in its selection, namely, by choosing a model that fits the data and is also succinct. Two-part codes and the minimum description length have been successful in delivering procedures to single out the best models, avoiding overfitting. In this work, we pursue this approach and complement it by performing further assumptions in the parameter space. Concretely, we assume that the parameter space is a smooth manifold, and by using tools of Riemannian geometry, we derive a sharper expression than the standard one given by the stochastic complexity, where the scalar curvature of the Fisher information metric plays a dominant role. Furthermore, we compute a sharper approximation to the capacity for exponential families and apply our results to derive optimal dimensional reduction in the context of principal component analysis.

The Impact of Functional Form Complexity on Model Overfitting for Nonlinear Mixed-Effects Models

Nonlinear mixed-effects models (NLMEMs) allow researchers to model curvilinear patterns of growth, but there is ambiguity as to what functional form the data follow. Often, researchers fit multiple nonlinear functions to data and use model selection criteria to decide which functional form fits the data “best.” Frequently used model selection criteria only account for the number of parameters in a model but overlook the complexity of intrinsically nonlinear functional forms. This can lead to overfitting and hinder the generalizability and reproducibility of results. The primary goal of this study was to evaluate the performance of eight model selection criteria via a Monte Carlo simulation study and assess under what conditions these criteria are sensitive to model overfitting as it relates to functional form complexity. Results highlighted criteria with the potential to capture overfitting for intrinsically nonlinear functional forms for NLMEMs. Information criteria and the stochastic information complexity criterion recovered the true model more often than the average or conditional concordance correlation. Results also suggest that the amount of residual variance and sample size have an impact on model selection for NLMEMs. Implications for future research and recommendations for application are also provided.

AIT Foundations of Structured Experience

In this paper, we present a unifying framework to study consciousness based on algorithmic information theory (AIT). We take as a premise that “there is experience” and focus on the requirements for structured experience ([Formula: see text]) — the spatial, temporal, and conceptual organization of our first-person experience of the world and of ourselves as agents in it. Our starting point is the insight that access to good models — succinct and accurate generative programs of world data — is crucial for homeostasis and survival. We hypothesize that the successful comparison of such models with data provides the structure to experience. Building on the concept of Kolmogorov complexity, we can associate the qualitative aspects of [Formula: see text] with the algorithmic features of the model, including its length, which reflects the structure discovered in the data. Moreover, a modeling system tracking structured data will display dimensionality reduction and criticality features that can be used empirically to quantify the structure of the program run by the agent. KT provides a consistent framework to define the concepts of life and agent and allows for the comparison between artificial agents and [Formula: see text]-reporting humans to provide an educated guess about agent experience. A first challenge is to show that a human agent has [Formula: see text] to the extent they run encompassing and compressive models tracking world data. For this, we propose to study the relation between the structure of neurophenomenological, physiological, and behavioral data. The second is to endow artificial agents with the means to discover good models and study their internal states and behavior. We relate the algorithmic framework to other theories of consciousness and discuss some of its epistemological, philosophical, and ethical aspects.

Exploration of stochastic dynamics and complexity in an epidemic system.

We investigate the effect of noise in an epidemic system. We have studied dynamics and complexity for both the deterministic and its noise-induced model. We have verified the stochastic sensitivity under the variation of noise strength and changing the initial conditions of the noise-induced system. It confirms that noise can make significant perturbation in the stochastic sensitivity. To quantify the dynamics, phase space analysis is done under both noisy and noise free conditions. The transition between regular and chaotic dynamics has been examined by 0--1 test. Corresponding complexity analysis is also done using the weighted recurrence entropy method. Numerical results confirm the chaotic dynamics in the noise-induced epidemic system within a larger region of parameters compared to the same in its noise free part.

Reversible Computations of One-Way Counter Automata

Deterministic one-way time-bounded multi-counter automata are studied with respect to their ability to perform reversible computations, which means that the automata are also backward deterministic and, thus, are able to uniquely step the computation back and forth. We study the computational capacity of such devices and obtain separation results between irreversible and reversible k-counter automata for superpolynomial time. For exponential time we obtain moreover an infinite and tight hierarchy with respect to the number of counters. This hierarchy is shown with Kolmogorov complexity and incompressibility arguments. In this way, on passing we can prove this hierarchy also for ordinary counter automata. This improves the known hierarchy for ordinary counter automata in the sense that here we consider a weaker acceptance condition. Then, it turns out that k+1 reversible counters are not better than k ordinary counters and vice versa. Finally, almost all usually studied decidability questions turn out to be undecidable and not even semidecidable for reversible multi-counter automata, if at least two counters are provided.