Calculating Kolmogorov complexity from the output frequency distributions of small Turing machines.

Fernando Soler-Toscano,Jean-Paul Delahaye,Hector Zenil,Nicolas Gauvrit,Matthias Dehmer

doi:10.1371/journal.pone.0096223

Abstract

Drawing on various notions from theoretical computer science, we present a novel numerical approach, motivated by the notion of algorithmic probability, to the problem of approximating the Kolmogorov-Chaitin complexity of short strings. The method is an alternative to the traditional lossless compression algorithms, which it may complement, the two being serviceable for different string lengths. We provide a thorough analysis for all binary strings of length and for most strings of length by running all Turing machines with 5 states and 2 symbols ( with reduction techniques) using the most standard formalism of Turing machines, used in for example the Busy Beaver problem. We address the question of stability and error estimation, the sensitivity of the continued application of the method for wider coverage and better accuracy, and provide statistical evidence suggesting robustness. As with compression algorithms, this work promises to deliver a range of applications, and to provide insight into the question of complexity calculation of finite (and short) strings.Additional material can be found at the Algorithmic Nature Group website at http://www.algorithmicnature.org. An Online Algorithmic Complexity Calculator implementing this technique and making the data available to the research community is accessible at http://www.complexitycalculator.com.

Highlights

The evaluation of the complexity of finite sequences is key in many areas of science
This paper provides an approximation to the output frequency distribution of all Turing machines with 5 states and 2 symbols which in turn allow us to apply a central theorem in the theory of algorithmic complexity based in the notion of algorithmic probability that relates frequency of production of a string and its Kolmogorov complexity providing, upon application of the theorem, numerical estimations of Kolmogorov complexity by a method different to lossless compression algorithms
We have put forward a method based on algorithmic probability that produces frequency distributions based on the production of strings using a standard (Rado’s) model of Turing machines generally used for the Busy Beaver problem

Summary

Introduction

The evaluation of the complexity of finite sequences is key in many areas of science. Researchers have for a long time avoided any practical use of the current accepted mathematical theory of randomness, mainly because it has been considered to be useless in practice [8]. Despite this belief, related notions such as lossless uncompressibility tests have proven relative success, in areas such as sequence pattern detection [21] and have motivated distance measures and classification methods [9] in several areas (see [19] for a survey), to mention but two examples among many others of even more practical use. This paper provides an approximation to the output frequency distribution of all Turing machines with 5 states and 2 symbols which in turn allow us to apply a central theorem in the theory of algorithmic complexity based in the notion of algorithmic probability ( known as Solomonoff’s theory of inductive inference) that relates frequency of production of a string and its Kolmogorov complexity providing, upon application of the theorem, numerical estimations of Kolmogorov complexity by a method different to lossless compression algorithms

Objectives

Methods

Results

Conclusion