Kolmogorov Complexity Research Articles

Kolmogorov complexity as intrinsic entropy of a pure state: Perspective from entanglement in free fermion systems

We consider free fermion systems in arbitrary dimensions and represent the occupation pattern of each eigenstate as a classical binary string. We find that the Kolmogorov complexity of the string correctly captures the scaling behavior of its entanglement entropy (EE). In particular, the logarithmically-enhanced area law for EE in the ground state and the volume law for EE in typical highly excited states are reproduced. Since our approach does not require bipartitioning the system, it allows us to distinguish typical and atypical eigenstates directly by their intrinsic complexity. We reveal that the fraction of atypical eigenstates which do not thermalize in the free fermion system vanishes exponentially in the thermodynamic limit. Our results illustrate explicitly the connection between complexity and EE of individual pure states in quantum systems.

Non-perturbative graph languages, halting problem and complexity

Abstract We explain the foundations of a new class of formal languages for the construction of large Feynman diagrams which contribute to solutions of all combinatorial Dyson–Schwinger equations in a given strongly coupled gauge field theory. Then we build a new Hopf algebraic structure on non-perturbative production rules which leads us to formulate the halting problem for the corresponding replacing–gluing graph grammars in our formal graph languages on the basis of Manin’s renormalization Hopf algebra. In addition, we apply topology of graphons to associate a complexity parameter to this new class of graph grammars. At the final step, we address some applications of our new formal language platform to Quantum Field Theory. The first application concerns the constructive role of non-perturbative graph languages in dealing with quantum gauge symmetries in the context of the Hopf ideals generated by Slavnov–Taylor or Ward–Takahashi identities. The second application concerns the importance of the complexities of non-perturbative replacing–gluing graph grammars in formulating a new generalization of the circuit complexity on the space of Dyson–Schwinger equations. We provide a geometric interpretation of non-perturbative circuit complexities. The third application concerns the impact of non-perturbative replacing–gluing graph grammars in providing some new tools for the computation of the Kolmogorov complexity of Dyson–Schwinger equations.

Does the “Delta Variant” affect the nonlinear dynamic characteristics of SARS-CoV-2 transmission?

In this paper, we analyzed the difference of nonlinear dynamic characteristics of SARS-CoV-2 transmission caused by ‘Delta Variant’. We selected the daily new diagnostic data of SARS-CoV-2 from 15 countries. Four different kinds of complexity metrics such as Kolmogorov complexity, Higuchi's Hurst exponent, Shannon entropy, and multifractal degrees were selected to explore the features of information content, persistence, randomness, multifractal complexity. Afterwards, Student's t-tests were performed to assess the presence of differences of these nonlinear dynamic characteristics for periods before and after “Delta Variant” appearance. The results of two-tailed Student's t-test showed that for all the nonlinear dynamic characteristics, the null hypothesis of equality of mean values were strongly rejected for the two periods. In addition, by one-tailed Student's t-test, we concluded that time series in “Delta period” exhibit higher value of Kolmogorov complexity and Shannon entropy, indicating a higher level of information content and randomness. On the other hand, the Higuchi's Hurst exponent in “Delta period” was lower, which showed the weaker persistent in this period. Moreover, the multifractal specturm width after “Delta” emergence were reduced, representing a more stable multifractality. The sources for the formation of multifractal features are also investigated.

A Dilemma for Solomonoff Prediction

AbstractThe framework of Solomonoff prediction assigns prior probability to hypotheses inversely proportional to their Kolmogorov complexity. There are two well-known problems. First, the Solomonoff prior is relative to a choice of universal Turing machine. Second, the Solomonoff prior is not computable. However, there are responses to both problems. Different Solomonoff priors converge with more and more data. Further, there are computable approximations to the Solomonoff prior. I argue that there is a tension between these two responses. This is because computable approximations to Solomonoff prediction do not always converge.

Stochastic Methods and Complexity Science in Climate Research and Modeling

The 2021 Nobel prize for physics was awarded to two climate scientists, Syukuro Manabe and Klaus Hasselmann, and the physicist Giorgio Parisi. While at first sight the work of Parisi seems not to be related to climate science, this is not the case. Giorgio Parisi developed and contributed to many complexity science methods which are nowadays widely used in climate science. Giorgi Parisi also was involved in the development of the “stochastic resonance” idea to explain paleoclimate variability, while Klaus Hasselmann developed stochastic climate models. Here we review and discuss their work from a complex and stochastic systems perspective in order to highlight those aspects of their work. For instance, fractal and multi-fractal analysis of climate data is now widely used and many weather prediction and climate models contain stochastic parameterizations, topics Parisi and Hasselmann have pioneered. Furthermore, Manabe’s work was key to understanding the effects of anthropogenic climate change by the development of key advances in the parameterization of convection and radiative forcing in climate models. We discuss also how their inventive research has shaped current climate research and is still influencing climate modeling and future research directions.

Living wearables: Bacterial reactive glove

A reactive bacterial glove is a cotton glove colonised by Acetobacter aceti, an example of biofabrication of a living electronic sensing device. The bacterial colony, supported by a cellulose-based hydrogel, forms a several millimetres-thick living coating on the surface of the glove. This paper proposes a novel method for analysing the complex electrical activity of trains of spikes generated by a living colony. The proposed method, which primarily focuses on dynamic entropy analysis, shows that the bacterial glove responds to mechanical triaxial stimuli by producing travelling patterns of electrical activity. Kolmogorov complexity further supports our investigation into the evolution of dynamic patterns of such waves in the hydrogel and shows how stimuli initiate electrical activity waves across the glove. These waves are diffractive and ultimately are suppressed by depression. Our experiments demonstrate that living substrates could be used to enable reactive sensing wearable by means of living colonies of bacteria, once the paradigm of excitation wave propagation and reflection is implemented.

Dimension spectra of lines1

This paper investigates the algorithmic dimension spectra of lines in the Euclidean plane. Given any line L with slope a and vertical intercept b, the dimension spectrum sp ( L ) is the set of all effective Hausdorff dimensions of individual points on L. We draw on Kolmogorov complexity and geometrical arguments to show that if the effective Hausdorff dimension dim ( a , b ) is equal to the effective packing dimension Dim ( a , b ), then sp ( L ) contains a unit interval. We also show that, if the dimension dim ( a , b ) is at least one, then sp ( L ) is infinite. Together with previous work, this implies that the dimension spectrum of any line is infinite.

An Application of Kolmogorov Complexity and Its Spectrum to Positive Surges

A positive surge is associated with a sudden change in flow that increases the water depth and modifies flow structure in a channel. Positive surges are frequently observed in artificial channels, rivers, and estuaries. This paper presents the application of Kolmogorov complexity and its spectrum to the velocity data collected during the laboratory investigation of a positive surge. Two types of surges were considered: a undular surge and a breaking surge. For both surges, the Kolmogorov complexity (KC) and Kolmogorov complexity spectrum (KCS) were calculated during the unsteady flow (US) associated with the passage of the surge as well as in the preceding steady-state (SS) flow condition. The results show that, while in SS, the vertical distribution of KC for Vx is dominated by the distance from the bed, with KC being the largest at the bed and the lowest at the free surface; in US only the passage of the undular surge was able to drastically modify such vertical distribution of KC resulting in a lower and constant randomness throughout the water depth. The analysis of KCS revealed that Vy values were peaking at about zero, while the distribution of Vx values was related both to the elevation from the bed and to the surge type. A comparative analysis of KC and normal Reynold stresses revealed that these metrics provided different information about the changes observed in the flow as it moves from a steady-state to an unsteady-state due to the surge passage. Ultimately, this preliminary application of Kolmogorov complexity measures to a positive surge provides some novel findings about such intricate hydrodynamics processes.

Kolmogorov Complexity Analysis and Prediction Horizon of the Daily Erythemal Dose Time Series

Influenced by stratospheric total ozone column (TOC), cloud cover, aerosols, albedo, and other factors, levels of daily erythemal dose (Her) in a specific geographic region show significant variability in time and space. To investigate the degree of randomness and predictability of Her time series from ground-based observations in Novi Sad, Serbia, during the 2003–2012 time period, we used a set of information measures: Kolmogorov complexity, Kolmogorov complexity spectrum, running Kolmogorov complexity, the largest Lyapunov exponent, Lyapunov time, and Kolmogorov time. The result reveals that fluctuations in daily Her are moderately random and exhibit low levels of chaotic behavior. We found a larger number of occurrences of deviation from the mean in the time series during the years with lower values of Her (2007–2009, 2011–2012), which explains the higher complexity. Our analysis indicated that the time series of daily values of Her show a tendency to increase the randomness when the randomness of cloud cover and TOC increases, which affects the short-term predictability. The prediction horizon of daily Her values in Novi Sad given by the Lyapunov time corrected for randomness by Kolmogorov is between 1.5 and 3.5 days.

Automatic Evaluation of Motor Rehabilitation Exercises Based on Deep Mixture Density Neural Networks

An automatic assessment system for physical telerehabilitation could reduce the time and cost of treatments. But such assessment involves stochastic uncertainties, nonlinearities, and complexities of human movement. Probabilistic models and deep structures are two categories that could, respectively, address the stochastic uncertainty and complexity of motion data. In the proposed Deep Mixture Density Network (DMDN), probabilistic models were concurrently processed along with deep neural networks. More specifically, a multi-branch convolutional layer extracted the deep features of motion data, a Long Short Term Memory (LSTM) learned its temporal dependencies, and a Gaussian Mixture Model (GMM) handled the stochastic interaction of its preceding layers in reaching a more valid assessment and improved generalization to new movements. Finally, the weighted mean of the GMM components was used as the performance score for exercises. Input data were the time series related to the joints' position and orientation extracted from the Kinect v2 sensor video. A clinical reference score for each movement was also included for training the DMDN. In addition to comparisons with the state-of-the-art algorithms, an ablation study of the various elements comprising the DMDN was performed. Three configurations of convolutional filter window transitions across input data were also investigated. Results indicate that the proposed DMDN with one-dimensional parallel window transitions outperforms the other competing strategies in the ablation study. It also achieves higher reliability in terms of a lower RMSE standard deviation against a DMDN without GMM strategy while ranking competitively according to the Spearman correlation coefficient and Root Mean Square Error.

Resource-bounded Kolmogorov complexity provides an obstacle to soficness of multidimensional shifts

We suggest necessary conditions for soficness of multidimensional shifts formulated in terms of resource-bounded Kolmogorov complexity. Using this technique we provide examples of effective and non-sofic shifts on Z2 with very low block complexity: the number of globally admissible patterns of size n×n grows only as a polynomial in n. We also show that more conventional proofs of non-soficness for multi-dimensional effective shifts, including the techniques of Pavlov (2013) [15] and Kass and Madden (2013) [6], can be expressed in terms of Kolmogorov complexity with unbounded computational resources.

Consistent Quantification of Complex Dynamics via a Novel Statistical Complexity Measure.

Natural systems often show complex dynamics. The quantification of such complex dynamics is an important step in, e.g., characterization and classification of different systems or to investigate the effect of an external perturbation on the dynamics. Promising routes were followed in the past using concepts based on (Shannon’s) entropy. Here, we propose a new, conceptually sound measure that can be pragmatically computed, in contrast to pure theoretical concepts based on, e.g., Kolmogorov complexity. We illustrate the applicability using a toy example with a control parameter and go on to the molecular evolution of the HIV1 protease for which drug treatment can be regarded as an external perturbation that changes the complexity of its molecular evolutionary dynamics. In fact, our method identifies exactly those residues which are known to bind the drug molecules by their noticeable signal. We furthermore apply our method in a completely different domain, namely foreign exchange rates, and find convincing results as well.

Classification of Depth of Coma Using Complexity Measures and Nonlinear Features of Electroencephalogram Signals.

In recent years, some electrophysiological analysis methods of consciousness have been proposed. Most of these studies are based on visual interpretation or statistical analysis, and there is hardly any work classifying the level of consciousness in a deep coma. In this study, we perform an analysis of electroencephalography complexity measures by quantifying features efficiency in differentiating patients in different consciousness levels. Several measures of complexity have been proposed to quantify the complexity of signals. Our aim is to lay the foundation of a system that will objectively define the level of consciousness by performing a complexity analysis of Electroencephalogram (EEG) signals. Therefore, a nonlinear analysis of EEG signals obtained with a recording scheme proposed by us from 39 patients with Glasgow Coma Scale (GCS) between 3 and 8 was performed. Various entropy values (approximate entropy, permutation entropy, etc.) obtained from different algorithms, Hjorth parameters, Lempel-Ziv complexity and Kolmogorov complexity values were extracted from the signals as features. The features were analyzed statistically and the success of features in classifying different levels of consciousness was measured by various classifiers. Consequently, levels of consciousness in deep coma (GCS between 3 and 8) were classified with an accuracy of 90.3%. To the authors' best knowledge, this is the first demonstration of the discriminative nonlinear features extracted from tactile and auditory stimuli EEG signals in distinguishing different GCSs of comatose patients.

Adaptive Lossless Image Data Compression Method Inferring Data Entropy by Applying Deep Neural Network

When we compress a large amount of data, we face the problem of the time it takes to compress it. Moreover, we cannot predict how effective the compression performance will be. Therefore, we are not able to choose the best algorithm to compress the data to its minimum size. According to the Kolmogorov complexity, the compression performances of the algorithms implemented in the available compression programs in the system differ. Thus, it is impossible to deliberately select the best compression program before we try the compression operation. From this background, this paper proposes a method with a principal component analysis (PCA) and a deep neural network (DNN) to predict the entropy of data to be compressed. The method infers an appropriate compression program in the system for each data block of the input data and achieves a good compression ratio without trying to compress the entire amount of data at once. This paper especially focuses on lossless compression for image data, focusing on the image blocks. Through experimental evaluation, this paper shows the reasonable compression performance when the proposed method is applied rather than when a compression program randomly selected is applied to the entire dataset.

EEG-Driven Prediction Model of Oxcarbazepine Treatment Outcomes in Patients With Newly-Diagnosed Focal Epilepsy.

Objective: Antiseizure medicine (ASM) is the first choice for patients with epilepsy. The choice of ASM is determined by the type of epilepsy or epileptic syndrome, which may not be suitable for certain patients. This initial choice of a particular drug affects the long-term prognosis of patients, so it is critical to select the appropriate ASMs based on the individual characteristics of a patient at the early stage of the disease. The purpose of this study is to develop a personalized prediction model to predict the probability of achieving seizure control in patients with focal epilepsy, which will help in providing a more precise initial medication to patients.Methods: Based on response to oxcarbazepine (OXC), enrolled patients were divided into two groups: seizure-free (52 patients), not seizure-free (NSF) (22 patients). We created models to predict patients' response to OXC monotherapy by combining Electroencephalogram (EEG) complexities and 15 clinical features. The prediction models were gradient boosting decision tree-Kolmogorov complexity (GBDT-KC) and gradient boosting decision tree-Lempel-Ziv complexity (GBDT-LZC). We also constructed two additional prediction models, support vector machine-Kolmogorov complexity (SVM-KC) and SVM-LZC, and these two models were compared with the GBDT models. The performance of the models was evaluated by calculating the accuracy, precision, recall, F1-score, sensitivity, specificity, and area under the curve (AUC) of these models.Results: The mean accuracy, precision, recall, F1-score, sensitivity, specificity, AUC of GBDT-LZC model after five-fold cross-validation were 81%, 84%, 91%, 87%, 91%, 64%, 81%, respectively. The average accuracy, precision, recall, F1-score, sensitivity, specificity, AUC of GBDT-KC model with five-fold cross-validation were 82%, 84%, 92%, 88%, 83%, 92%, 83%, respectively. We used the rank of absolute weights to separately calculate the features that have the most significant impact on the classification of the two models.Conclusion: (1) The GBDT-KC model has the potential to be used in the clinic to predict seizure-free with OXC monotherapy. (2). Electroencephalogram complexity, especially Kolmogorov complexity (KC) may be a potential biomarker in predicting the treatment efficacy of OXC in newly diagnosed patients with focal epilepsy.

The Relationship between Population Size, Technology Complexity, Economic Growth and Kolmogorov Complexity

The Relationship between Population Size, Technology Complexity, Economic Growth and Kolmogorov Complexity

On Kolmogorov Complexity of Unitary Transformations in Quantum Computing

We introduce a notion of Kolmogorov complexity of unitary transformation, which can (roughly) be understood as the least possible amount of information required to fully describe and reconstruct a given finite unitary transformation. In the context of quantum computing, it corresponds to the least possible amount of data to define and describe a quantum circuit or quantum computer program. Our Kolmogorov complexity of unitary transformation is built upon Kolmogorov "qubit complexity" of Berthiaume, W. Van Dam and S. Laplante via mapping from unitary transformations to unnormalized density operators, which are subsequently "purified" into unnormalized vectors in Hilbert space. We discuss the optimality of our notion of Kolmogorov complexity in a broad sense.

An innovative and robust technique for human identification and authentication based on a secure clinical signals transmission

EEG (Electroencephalogram) is brain waves measure. It is available test allowed to discover the brain functions over time. The brain troubles are evaluated by EEG. It is used to locate the activity in the brain during a seizure and to consider the patients who suffer from brain functionality problems. These troubles include tumors, coma, confusion and long-term difficulties (such as weakness associated with a stroke). The acquisition of EEG signals requires contact and liveliness and these signals are changes under stress that make so potentially unnecessary if it is acquired under menace. In this paper, an innovative and robust solution for this problem is introduced. To this end, the manner depends on models of various data compression models of information-theoretic plus the metrics symmetry related to Kolmogorov complexity. The proposed procedure compares two EEG segments and clusters the data into three groups: a corresponding record for each participant, a distinct person for each group, and self-participant. The technique was used to determine the database participant based on EEG signals. Using a distance measuring approach suggested in this scheme, a 1-NN classifier was constructed. Nearly every person in the underlying database could be accurately identified by the classifier with $96%$ accuracy

Recursion in programs, thought, and language

This article presents a theory of recursion in thinking and language. In the logic of computability, a function maps one or more sets to another, and it can have a recursive definition that is semi-circular, i.e., referring in part to the function itself. Any function that is computable - and many are not - can be computed in an infinite number of distinct programs. Some of these programs are semi-circular too, but they needn't be, because repeated loops of instructions can compute any recursive function. Our theory aims to explain how naive individuals devise informal programs in natural language, and is itself implemented in a computer program that creates programs. Participants in our experiments spontaneously simulate loops of instructions in kinematic mental models. They rely on such loops to compute recursive functions for rearranging the order of cars in trains on a track with a siding. Kolmogorov complexity predicts the relative difficulty of abducing such programs - for easy rearrangements, such as reversing the order of the cars, to difficult ones, such as splitting a train in two and interleaving the two resulting halves (equivalent to a faro shuffle). This rearrangement uses both the siding and part of the track as working memories, shuffling cars between them, and so it relies on the power of a linear-bounded computer. Linguistic evidence implies that this power is more than necessary to compose the meanings of sentences in natural language from those of their grammatical constituents.

The causes of difficulty in children’s creation of informal programs

We present a theory of the causes of difficulty in children’s creation of informal programs. Ten-year-old children are able to devise such programs to rearrange the order of the cars in trains on a simple railway track with a single siding. According to the theory, they rely on kinematic mental models that simulate the required sequence of steps, and we devised a computer program, mAbducer, which does so too in creating its own programs for such rearrangements. An experiment showed that a simple measure of the complexity of its programs, based on Kolmogorov complexity, predicts ten-year-olds’ difficulty in this task: the measure is the number of words in mAbducer’s programs for solving the rearrangement in a minimal number of moves. Complexity, in turn, reflects the structure of the required programs, which need loops of moves to be repeated, and often moves before and after such a loop. Children’s errors are predictable in both their location and nature. Our results therefore have implications for the assessment and pedagogy of computational thinking.