Approximation methods in inductive inference

Abstract

In many areas of scientific inquiry, the phenomena under investigation are viewed as functions on the real numbers. Since observational precision is limited, it makes sense to view these phenomena as bounded functions on the rationals. One may translate the basic notions of recursion theory into this framework by first interpreting a partial recursive function as a function on Q. The standard notions of inductive inference carry over as well, with no change in the theory. When considering the class of computable functions on Q, there are a number of natural ways in which to define the distance between two functions. We utilize standard metrics to explore notions of approximate inference — our inference machines will attempt to guess values which converge to the correct answer in these metrics. We show that the new inference notions, NV ∞, EX ∞, and BC ∞, infer more classes of functions than their standard counterparts, NV, EX, and BC. Furthermore, we give precise inclusions between the new inference notions and those in the standard inference hierarchy. We also explore weaker notions of approximate inference, leading to inference hierarchies analogous to the EX n and BC n hierarchies. Oracle inductive inference is also considered, and we give sufficient conditions under which approximate inference from a generic oracle G is equivalent to approximate inference with only finitely many queries to G.