Kolmogorov Complexity Measure Research Articles

Utility of Kolmogorov complexity measures: Analysis of L2 groups and L1 backgrounds.

The proliferation of automated syntactic complexity tools allowed the analysis of larger amounts of learner writing. However, existing tools tend to be language-specific or depend on segmenting learner production into native-based units of analysis. This study examined the utility of a language-general and unsupervised linguistic complexity metric: Kolmogorov complexity in discriminating between L2 proficiency levels within several languages (Czech, German, Italian, English) and across various L1 backgrounds (N = 10) using two large CEFR-rater learner corpora. Kolmogorov complexity was measured at three levels: syntax, morphology, and overall linguistic complexity. Pairwise comparisons indicated that all Kolmogorov complexity measures discriminated among the proficiency levels within the L2s. L1-based variation in complexity was also observed. Distinct syntactic and morphological complexity patterns were found when L2 English writings were analyzed across versus within L1 backgrounds. These results indicate that Kolmogorov complexity could serve as a valuable metric in L2 writing research due to its cross-linguistic flexibility and holistic nature.

An Application of Kolmogorov Complexity and Its Spectrum to Positive Surges

A positive surge is associated with a sudden change in flow that increases the water depth and modifies flow structure in a channel. Positive surges are frequently observed in artificial channels, rivers, and estuaries. This paper presents the application of Kolmogorov complexity and its spectrum to the velocity data collected during the laboratory investigation of a positive surge. Two types of surges were considered: a undular surge and a breaking surge. For both surges, the Kolmogorov complexity (KC) and Kolmogorov complexity spectrum (KCS) were calculated during the unsteady flow (US) associated with the passage of the surge as well as in the preceding steady-state (SS) flow condition. The results show that, while in SS, the vertical distribution of KC for Vx is dominated by the distance from the bed, with KC being the largest at the bed and the lowest at the free surface; in US only the passage of the undular surge was able to drastically modify such vertical distribution of KC resulting in a lower and constant randomness throughout the water depth. The analysis of KCS revealed that Vy values were peaking at about zero, while the distribution of Vx values was related both to the elevation from the bed and to the surge type. A comparative analysis of KC and normal Reynold stresses revealed that these metrics provided different information about the changes observed in the flow as it moves from a steady-state to an unsteady-state due to the surge passage. Ultimately, this preliminary application of Kolmogorov complexity measures to a positive surge provides some novel findings about such intricate hydrodynamics processes.

Randomness in flow turbulence around a bridge pier in a sand mined channel

Mining activities have the potential to cause excess scour and affect the streambed stability around bridge piers present in the vicinity. Randomness characteristics of turbulence around the pier in a mining infested channel can be helpful in better understanding of this phenomenon. In this study, the effect of a mining pit on the randomness in turbulent flow around the pier has been explored on the basis of Kolmogorov complexity (KC) measure and its derivative, i.e. KC spectrum. Conventionally, turbulent stresses are supposed to govern the sediment transport characteristics, hence the Reynolds shear stresses (RSS) and bursting events are also analyzed. Results of two cases have been comparatively studied; for the first case, only an oblong pier is embedded in the channel while in the second case, a rectangular mining pit is excavated upstream of the pier. Quadrant analysis of the Reynolds shear stress revealed that strong ejection events in the upstream side and inward interaction events in the downstream side of the pier cause excess momentum transport in presence of a mining pit. Streamwise and vertical turbulence intensities also increase by 20% and 33% respectively in the down-flow zone in front of the pier. KC value of turbulence around the pier, especially in the down-flow zone and separation zone is found to be 10% and 12% higher due to pit excavation. Comparing the KC spectra for both cases suggests that dredging of a pit in the channel increases the degree of randomness of higher velocity scales around the bridge pier. These alterations in turbulent stresses and randomness characteristics can be helpful in explaining the streambed scour and undermining of bridge piers due to mining activities.

Analysis of daily streamflow complexity by Kolmogorov measures and Lyapunov exponent

Analysis of daily streamflow variability in space and time is important for water resources planning, development, and management. The natural variability of streamflow is being complicated by anthropogenic influences and climate change, which may introduce additional complexity into streamflow records. To address the complexity in streamflow, daily discharge data recorded during the period 1989–2016 at twelve gauging stations on Brazos River in Texas (USA) were used to derive a set of novel quantitative tools: Kolmogorov complexity (KC) and its derivative-associated measures to assess complexity, and Lyapunov time (LT) to assess predictability. It was found that all daily discharge series exhibited long memory with an increasing down-flow tendency, while the randomness of the series at individual sites could not be definitively concluded. All Kolmogorov complexity measures had relatively small values with the exception of the USGS (United States Geological Survey) 08088610 station at Graford, Texas, which exhibited the highest values of the complexity measures. This finding may be attributed to the elevated effect of human activities at Graford, and proportionally lesser effect at other stations. In addition, complexity tended to decrease downflow, meaning that larger catchments were generally less influenced by anthropogenic activities. The correction on randomness of Lyapunov time (quantifying predictability) was found to be inversely proportional to the Kolmogorov complexity, which strengthened our conclusion regarding the effect of anthropogenic activities, considering that KC and LT were distinct measures, based on rather different techniques.

Randomness Representation of Turbulence in Canopy Flows Using Kolmogorov Complexity Measures

Turbulence is often expressed in terms of either irregular or random fluid flows, without quantification. In this paper, a methodology to evaluate the randomness of the turbulence using measures based on the Kolmogorov complexity (KC) is proposed. This methodology is applied to experimental data from a turbulent flow developing in a laboratory channel with canopy of three different densities. The methodology is even compared with the traditional approach based on classical turbulence statistics.

The Minimum Oracle Circuit Size Problem

We consider variants of the minimum circuit size problem MCSP, where the goal is to minimize the size of oracle circuits computing a given function. When the oracle is QBF, the resulting problem MSCP QBF is known to be complete for PSPACE under ZPP reductions. We show that it is not complete under logspace reductions, and indeed it is not even hard for TC 0 under uniform AC 0 reductions. We obtain a variety of consequences that follow if oracle versions of MCSP are hard for various complexity classes under different types of reductions. We also prove analogous results for the problem of determining the resource-bounded Kolmogorov complexity of strings, for certain types of Kolmogorov complexity measures.

Rolling bearing quality evaluation based on a morphological filter and a Kolmogorov complexity measure

Bearing defective inspection plays a vital role in bearing quality control. Unlike signals in the process of condition monitoring and fault diagnosis, the signal characteristic of defective bearings is much weaker and difficult to be quantified through the acceleration based techniques. In this paper, a novel system is developed to inspect automatically the small defects of roller bearings for on-line quality control. Rather than using acceleration based techniques the system employs a high sensitive eddy current sensor to measure the displacement profiles of the outer race for high signal to noise ratio. Furthermore, a morphological filter is used to enhance the feature signal which is subsequently measured by Kolmogorov complexity measure. Both simulated signals and measured data show that this system is able to diagnose defects including abnormal surface roundness, waviness, misaligned races which are typical quality problems in bearing manufacturing lines.

Maximizing Kolmogorov Complexity for accurate and robust bright field cell segmentation

BackgroundAnalysis of cellular processes with microscopic bright field defocused imaging has the advantage of low phototoxicity and minimal sample preparation. However bright field images lack the contrast and nuclei reporting available with florescent approaches and therefore present a challenge to methods that segment and track the live cells. Moreover, such methods must be robust to systemic and random noise, variability in experimental configuration, and the multiple unknowns in the biological system under study.ResultsA new method called maximal-information is introduced that applies a non-parametric information theoretic approach to segment bright field defocused images. The method utilizes a combinatorial optimization strategy to select specific defocused images from each image stack such that set complexity, a Kolmogorov complexity measure, is maximized. Differences among these selected images are then applied to initialize and guide a level set based segmentation algorithm. The performance of the method is compared with a recent approach that uses a fixed defocused image selection strategy over an image data set of embryonic kidney cells (HEK 293T) from multiple experiments. Results demonstrate that the adaptive maximal-information approach significantly improves precision and recall of segmentation over the diversity of data sets.ConclusionsIntegrating combinatorial optimization with non-parametric Kolmogorov complexity has been shown to be effective in extracting information from microscopic bright field defocused images. The approach is application independent and has the potential to be effective in processing a diversity of noisy and redundant high throughput biological data.

KOLMOGOROV COMPLEXITY SPECTRUM FOR USE IN ANALYSIS OF UV-B RADIATION TIME SERIES

In this paper, we have used the Kolmogorov complexity and sample entropy measures to estimate the complexity of the UV-B radiation time series in the Vojvodina region (Serbia) for the period 1990–2007. We have defined the Kolmogorov complexity spectrum and have introduced the Kolmogorov complexity spectrum highest value (KCH). We have established the UV-B radiation time series on the basis of their daily sum (dose) for seven representative places in this region using: (i) measured data, (ii) data calculated via a derived empirical formula and (iii) data obtained by a parametric UV radiation model. We have calculated the Kolmogorov complexity (KC) based on the Lempel–Ziv algorithm (LZA), KCH and sample entropy (SE) values for each time series. We have divided the period 1990–2007 into two subintervals: (i) 1990–1998 and (ii) 1999–2007 and calculated the KC, KCH and SE values for the various time series in these subintervals. It is found that during the period 1999–2007, there is a decrease in the KC, KCH and SE, compared to the period 1990–1998. This complexity loss may be attributed to (i) the increased human intervention in the post civil war period causing increase of the air pollution and (ii) the increased cloudiness due to climate changes.

The pervasive reach of resource-bounded Kolmogorov complexity in computational complexity theory

We continue an investigation into resource-bounded Kolmogorov complexity (Allender et al., 2006 [4]), which highlights the close connections between circuit complexity and Levin's time-bounded Kolmogorov complexity measure Kt (and other measures with a similar flavor), and also exploits derandomization techniques to provide new insights regarding Kolmogorov complexity. The Kolmogorov measures that have been introduced have many advantages over other approaches to defining resource-bounded Kolmogorov complexity (such as much greater independence from the underlying choice of universal machine that is used to define the measure) (Allender et al., 2006 [4]). Here, we study the properties of other measures that arise naturally in this framework. The motivation for introducing yet more notions of resource-bounded Kolmogorov complexity are two-fold: • to demonstrate that other complexity measures such as branching-program size and formula size can also be discussed in terms of Kolmogorov complexity, and • to demonstrate that notions such as nondeterministic Kolmogorov complexity and distinguishing complexity (Buhrman et al., 2002 [15]) also fit well into this framework. The main theorems that we provide using this new approach to resource-bounded Kolmogorov complexity are: • A complete set ( R KNt ) for NEXP / poly defined in terms of strings of high Kolmogorov complexity. • A lower bound, showing that R KNt is not in NP ∩ coNP . • New conditions equivalent to the conditions “ NEXP ⊆ nonuniform NC 1 ” and “ NEXP ⊆ L / poly ”. • Theorems showing that “distinguishing complexity” is closely connected to both FewEXP and to EXP. • Hardness results for the problems of approximating formula size and branching program size.

On the computational power of random strings

There are two fundamental computably enumerable sets associated with any Kolmogorov complexity measure. These are the set of non-random strings and the overgraph. This paper investigates the computational power of these sets. It follows work done by Kummer, Muchnik and Positselsky, and Allender and co-authors. Muchnik and Positselsky asked whether there exists an optimal monotone machine whose overgraph is not t t -complete. This paper answers this question in the negative by proving that the overgraph of any optimal monotone machine, or any optimal process machine, is t t -complete. The monotone results are shown for both descriptional complexity K m and K M , the complexity measure derived from algorithmic probability. A distinction is drawn between two definitions of process machines that exist in the literature. For one class of process machines, designated strict process machines, it is shown that there is a universal machine whose set of non-random strings is not t t -complete.

Is Entropy Suitable to Characterize Data and Signals for Cognitive Informatics?

This article provides a review of Shannon and other entropy measures in evaluating the quality of materials used in perception, cognition, and learning processes. Energy-based metrics are not suitable for cognition, as energy itself does not carry information. Instead, morphological (structural and contextual) metrics as well as entropy-based multi-scale metrics should be considered in cognitive informatics. Appropriate data and signal transformation processes are defined and discussed in the perceptual framework followed by various classes of information and entropies suitable for characterization of data, signals, and distortion. Other entropies are also described including the Rényi generalized entropy spectrum, Kolmogorov complexity measure, Kolmogorov-Sinai entropy, and Prigogine entropy for evolutionary dynamical systems. Although such entropy-based measures are suitable for many signals, they are not sufficient for scale-invariant (fractal and multifractal) signals without corresponding complementary multi-scale measures.

Power from Random Strings

We show that sets consisting of strings of high Kolmogorov complexity provide examples of sets that are complete for several complexity classes under probabilistic and nonuniform reductions. These sets are provably not complete under the usual many-one reductions. Let ${{R_{\rm C}}}, {{R_{\rm Kt}}}, {{R_{\rm KS}}}, {{R_{\rm KT}}}$ be the sets of strings x having complexity at least $|x|/2$, according to the usual Kolmogorov complexity measure ${\mbox{\rm C}}$, Levin's time-bounded Kolmogorov complexity ${\mbox{\rm Kt}}$ [L. Levin, Inform. and Control, 61 (1984), pp. 15-37], a space-bounded Kolmogorov measure ${\mbox{\rm KS}}$, and a new time-bounded Kolmogorov complexity measure ${\mbox{\rm KT}}$, respectively. Our main results are as follows: \begin{remunerate} \item ${{R_{\rm KS}}}$ and ${{R_{\rm Kt}}}$ are complete for ${{\rm{PSPACE}}}$ and {\mbox{\rm EXP}}, respectively, under ${\mbox{\rm P/poly}}$-truth-table reductions. Similar results hold for other classes with ${{\rm{PSPACE}}}$-robust Turing complete sets. \item ${\mbox{\rm EXP}} = {\mbox{\rm NP}}^{{{R_{\rm Kt}}}}.$ \item ${{\rm{PSPACE}}} = {\mbox{\rm ZPP}}^{{{R_{\rm KS}}}} \subseteq {\mbox{\rm P}}^{{{R_{\rm C}}}}$. \item The Discrete Log, Factoring, and several lattice problems are solvable in ${\mbox{\rm BPP}}^{{{R_{\rm KT}}}}$. \end{remunerate} Our hardness result for ${{\rm{PSPACE}}}$ gives rise to fairly natural problems that are complete for ${{\rm{PSPACE}}}$ under ${\mbox{$\leq^{\rm p}_{\rm T}$}}$ reductions, but not under ${\mbox{$\leq^{\rm log}_{\rm m}$}}$ reductions. Our techniques also allow us to show that all computably enumerable sets are reducible to ${{R_{\rm C}}}$ via ${\mbox{\rm P/poly}}$-truth-table reductions. This provides the first "efficient" reduction of the halting problem to ${{R_{\rm C}}}$.

Computational depth: Concept and applications

We introduce Computational Depth, a measure for the amount of “nonrandom” or “useful” information in a string by considering the difference of various Kolmogorov complexity measures. We investigate three instantiations of Computational Depth: • Basic Computational Depth, a clean notion capturing the spirit of Bennett's Logical Depth. We show that a Turing machine M runs in time polynomial on average over the time-bounded universal distribution if and only if for all inputs x, M uses time exponential in the basic computational depth of x. • Sublinear-time Computational Depth and the resulting concept of Shallow Sets, a generalization of sparse and random sets based on low depth properties of their characteristic sequences. We show that every computable set that is reducible to a shallow set has polynomial-size circuits. • Distinguishing Computational Depth, measuring when strings are easier to recognize than to produce. We show that if a Boolean formula has a nonnegligible fraction of its satisfying assignments with low depth, then we can find a satisfying assignment efficiently.

Hierarchical content classification and script determination for automatic document image processing

Page segmentation and image content classification play an important role in automatic image processing with applications to mixed-type document image compression, form and check reading, and automatic mail sorting. In this paper, we first present an enhanced background thinning based approach for fast page segmentation. After the analysis of three different methods individually, a hierarchical approach for document content classification is proposed, which classifies a sub-image into one of two categories: text and halftone. Our approach combines a neural network model, cross-correlation metric, and Kolmogorov complexity measure in a hierarchical structure. Considering the necessity of a recognition system, we also propose using a three-layer feedforward neural network to classify text regions into Chinese and English scripts. The classification accuracy on a number of document images reaches 100% and 97.1% for halftone region and text region, respectively. Meanwhile, the system can achieve a correct rate of 92.3% and 95.0% for Chinese and alphabetic script determination, respectively.

Random Strings Make Hard Instances

We establish the truth of the “instance complexity conjecture” in the case of DEXT-complete sets over polynomial time computations, and r.e. complete sets over recursive computations. Specifically, we obtain for every DEXT-complete setAan exponentially dense subsetCand a constantksuch that for every nondecreasing polynomialt(n)=Ω(nk), ict(x:A)⩾Kt(x)−cholds for some constantcand allx∈C, where ictandKtare thet-bounded instance complexity and Kolmogorov complexity measures, respectively. For any r.e. complete setAwe obtain an infinite setC⊆Āsuch that ic(x:A)⩾K(x)−cholds for some constantcand allx∈C, where ic andKdenote the time-unbounded versions of instance and Kolmogorov complexities, respectively. The proofs are based on the observation thatKolmogorov random stringsare individually hard to recognize by bounded computations.

Generalized kolmogorov complexity and other dual complexity measures

We construct and analyze some dual complexity measures that indicate the time it takes to obtain the desired object. The existence of optimal dual complexity measures is established. Various relations between dual measures and complexities are determined. The relationship of these measures to program quality is demonstrated.

Several results in program size complexity

Intuitively, the program size complexity of a binary string measures the amount of information in the string. Researchers have formalized this notion in a number of different ways. Here, we demonstrate similarities between some of these formulations. We also investigate in some detail the properties of Kolmogorov's complexity measure.