Complexity-based induction systems: Comparisons and convergence theorems

Abstract

In 1964 the author proposed as an explication of {\em a priori} probability the probability measure induced on output strings by a universal Turing machine with unidirectional output tape and a randomly coded unidirectional input tape. Levin has shown that if <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tilde{P}'_{M}(x)</tex> is an unnormalized form of this measure, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P(x)</tex> is any computable probability measure on strings, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</tex> , then <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\tilde{P}'_{M}\geqCP(x)</tex> where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</tex> is a constant independent of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</tex> . The corresponding result for the normalized form of this measure, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P'_{M}</tex> , is directly derivable from Willis' probability measures on nonuniversal machines. If the conditional probabilities of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P'_{M}</tex> are used to approximate those of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</tex> , then the expected value of the total squared error in these conditional probabilities is bounded by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-(1/2) \ln C</tex> . With this error criterion, and when used as the basis of a universal gambling scheme, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P'_{M}</tex> is superior to Cover's measure <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b\ast</tex> . When <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H\ast\equiv -\log_{2} P'_{M}</tex> is used to define the entropy of a rmite sequence, the equation <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H\ast(x,y)= H\ast(x)+H^{\ast}_{x}(y)</tex> holds exactly, in contrast to Chaitin's entropy definition, which has a nonvanishing error term in this equation.