Input\u2013output maps are strongly biased towards simple outputs

Kamaludin Dingle,Ard A Louis,Chico Q Camargo

doi:10.1038/s41467-018-03101-6

Kamaludin Dingle, Ard A Louis + Show 1 more

Open Access

https://doi.org/10.1038/s41467-018-03101-6

Copy DOI

Abstract

Many systems in nature can be described using discrete input–output maps. Without knowing details about a map, there may seem to be no a priori reason to expect that a randomly chosen input would be more likely to generate one output over another. Here, by extending fundamental results from algorithmic information theory, we show instead that for many real-world maps, the a priori probability P(x) that randomly sampled inputs generate a particular output x decays exponentially with the approximate Kolmogorov complexity tilde K(x) of that output. These input–output maps are biased towards simplicity. We derive an upper bound P(x) ≲ 2^{ - atilde K(x) - b}, which is tight for most inputs. The constants a and b, as well as many properties of P(x), can be predicted with minimal knowledge of the map. We explore this strong bias towards simple outputs in systems ranging from the folding of RNA secondary structures to systems of coupled ordinary differential equations to a stochastic financial trading model.

Highlights

Many systems in nature can be described using discrete input–output maps
We explicitly demonstrate the exponential bias towards low-complexity outputs in systems ranging from RNA folding to coupled ordinary differential equations to a financial trading model
Following a standard procedure from AIT7,10, each output x ∈ O can be described with the following algorithm: first enumerate all inputs using n and map these inputs to their outputs using the map f

Summary

Introduction

Many systems in nature can be described using discrete input–output maps. Without knowing details about a map, there may seem to be no a priori reason to expect that a randomly chosen input would be more likely to generate one output over another. By extending fundamental results from algorithmic information theory, we show instead that for many real-world maps, the a priori probability P(x) that randomly sampled inputs generate a particular output x decays exponentially with the approximate Kolmogorov complexity K~ ðxÞ of that output These input–output maps are biased towards simplicity. Examples include differential equations, where the inputs are discretised values of the equation parameters, and the outputs are discretised values of the solutions for a given set of boundary conditions Such a wide diversity of map types might at first sight suggest that, without knowing details of a particular map, there are no grounds for predicting one output to be more likely than another. We provide a short pedagogical description of AIT pertinent to the current paper in

Methods

Results

Conclusion