Inductive Inference Machines Research Articles

Quantum inductive inference by finite automata

Freivalds and Smith [R. Freivalds, C.H. Smith Memory limited inductive inference machines, Springer Lecture Notes in Computer Science 621 (1992) 19–29] proved that probabilistic limited memory inductive inference machines can learn with probability 1 certain classes of total recursive functions, which cannot be learned by deterministic limited memory inductive inference machines. We introduce quantum limited memory inductive inference machines as quantum finite automata acting as inductive inference machines. These machines, we show, can learn classes of total recursive functions not learnable by any deterministic, nor even by probabilistic, limited memory inductive inference machines.

Maximal Machine Learnable Classes

A class of computable functions ismaximaliff it can be incrementally learned by some inductive inference machine (IIM), but no infinitely larger class of computable functions can be so learned. Rolf Wiehagen posed the question whether there exist such maximal classes. This question and many interesting variants are answered herein in the negative. Viewed positively, each IIM can be infinitely improved upon! Also discussed are the problems of algorithmically finding the improvements proved to exist.

On the Structure of Degrees of Inferability

Degrees of inferability have been introduced to measure the learning power of inductive inference machines which have access to an oracle. The classical concept of degrees of unsolvability measures the computing power of oracles. In this paper we determine the relationship between both notions.

Towards a mathematical theory of machine discovery from facts

This paper intends to give a theoretical foundation of machine discovery from facts. We point out that the essence of a computational logic of scientific discovery or a logic of machine discovery is the refutability of the entire spaces of hypotheses. We discuss this issue in the framework of inductive inference of length-bounded elementary formal systems (EFSs), which are a kind of logic programs over strings of characters and correspond to context-sensitive grammars in Chomsky hierarchy. First we present some characterization theorems on inductive inference machines that can refute hypothesis spaces. Then we show differences between our inductive inference and some other related inferences such as in the criteria of reliable identification, finite identification and identification in the limit. Finally we show that for any n, the class, i.e. hypothesis space, of length-bounded EFSs with at most n axioms is inferable in our sense, that is, the class is refutable by a consistently working inductive inference machine. This means that sufficiently large hypothesis spaces are identifiable and refutable.

Extremes in the degrees of inferability

Most theories of learning consider inferring a function f from either (1) observations about f or, (2) questions about f. We consider a scenario whereby the learner observes f and asks queries to some set A. If I is a notion of learning then I[ A] is the set of concept classes I-learnable by an inductive inference machine with oracle A. A and B are I-equivalent if I[ A] = I[ B]. The equivalence classes induced are the degrees of inferability. We prove several results about when these degrees are trivial, and when the degrees are omniscient (i.e., the set of recursive function is learnable).

Aggregating inductive expertise on partial recursive functions

The notion of an inductive inference machine aggregating a team of inference machines models the problem of making use of several explanations for a single phenomenon. This article investigates the amount of information necessary for a successful aggregation of the theories given by a team of inference machines. Variations of using different kinds of identification and aggregation are investigated.

Learning in the presence of partial explanations

The effect of a partial explanation as additional information in the learning process is investigated. A scientist performs experiments to gather experimental data about some phenomenon, and then tries to construct an explanation (or theory) for the phenomenon. A plausible model for the practice of science is an inductive inference machine (scientist) learning a program (explanation) from a graph (set of experiments) of a recursive function (phenomenon). It is argued that this model of science is not an adequate one, as scientists, in addition to performing experiments, make use of some approximate partial explanation based on the “state of the art” knowledge about that phenomenon. An attempt has been made to model this partial explanation as additional information in the scientific process. It is shown that the inference capability of machines is improved in the presence of such a partial explanation. The quality of this additional information is modeled using certain “density” notions. It is shown that additional information about a “better” quality partial explanation enhances the inference capability of learning machines as scientists more than a “not so good” partial explanation. Similar enhancements to inference of approximations, a more sophisticated model of science, are demonstrated. Inadequacies in Gold's paradigm of language learning are investigated. It is argued that Gold's model fails to incorporate certain additional information that children get from their environment. Children are sometimes told about some grammatical rule that enumerates elements of the language. It is argued that these rules are a kind of additional information. They enable children to see in advance elements that are yet to appear in their environments. Also, children are being given some information about what is not in the language. Sometimes, they are rebuked for making incorrect utterances, or are told of a rule that enumerates certain non-elements of the language. An attempt has been made to extend Gold's model to incorporate both the above types of additional information. It is shown that either type of additional information enhances the learning capability of formal language learning devices.

Probabilistic inductive inference

Inductive inference machines construct programs for total recursive functions given only example values of the functions. Probabilistic inductive inference machines are defined, and for various criteria of successful inference, it is asked whether a probabilistic inductive inference machine can infer larger classes of functions if the inference criterion is relaxed to allow inference with probability at least p , (0 < p < 1) as opposed to requiring certainty. For the most basic criteria of success ( EX and BC ), it is shown that any class of functions that can be inferred from examples with probability exceeding 1/2 can be inferred deterministically, and that for probabilities p ≤ 1/2 there is a discrete hierarchy of inferability parameterized by p . The power of probabilistic inference strategies is characterized by equating the classes of probabilistically inferable functions with those classes that can be inferred by teams of inductive inference machines (a parallel model of inference), or by a third model called frequency inference.

Saving the phenomena: Requirements that inductive inference machines not contradict known data

Three kinds of restrictions on inductive inference machines (IIMs) are considered: postdictive completeness, postdictive consistency, and reliability. It is shown that postdictively consistent IIMs can be effectively replaced with post-dictively complete IIMs that succeed to at least the same degree. Various loosenings of the notions of postdictive completeness and reliability are considered, and a pair of related triangular hierarchies is exhibited; IIMs higher (or to the right) in the hierarchies are less restricted and capable of learning more than IIMs lower or to the left. Various conjectures and older results are obtained as corollaries.

Chaos dynamics executes inductive inference

A chaos dynamics model of inductive inference is proposed. Inductive inference is a process to predict the future from a finite length of a complex sequence of data. Chaos dynamics is able to generate a complex sequence of data governed by a deterministic rule. A certain deterministic rule can be assigned to a finite length of any complex sequence of data in reverse. Once a deterministic rule is assigned to this finite length of complex data, the rule will in turn generate another complex sequence of data in the future. This process of generating data in the future is equivalent to future prediction. The overall process of assigning a deterministic rule and generating data in the future, therefore, is considered to be an inductive inference process. This chaos dynamics model may provide an effective concept for theorizing inductive inference. It is also suggestive of future realization of an inductive inference machine simulating the information processing of the brain.

Aggregating inductive expertise

The aggregation problem is to design an inferential agent that makes intelligent use of the theories offered by a team of inductive inference machines working in a common environment. The present paper formulates several versions of the aggregation problem and investigates them from a recursion theoretic point of view.

The position of index sets of identifiable sets in the arithmetical hierarchy

We prove that every set of partial recursive functions which can be identified by an inductive inference machine is included in some identifiable function set with index set in Σ 3 ∩ Π 3 . An identifiable set is presented with index set in Σ 2 ∩ Π 3 but neither in Σ 2 nor in Π 2 . Furthermore we show that there is no nonempty identifiable set with index set in Σ 1 . In Π 1 it is possible to locate this king of set. In the last part of the paper we show that the problem to identify all partial recursive functions and the halting problem are of the same degree of unsolvability.

Comparison of identification criteria for machine inductive inference

A natural ω pLω+1 hierarchy of successively more general criteria of success for inductive inference machines is described based on the size of sets of anomalies in programs synthesized by such machines. These criteria are compared to others in the literature. Some of our results are interpreted as tradeoff results or as showing the inherent relative-computational complexity of certain processes and others are interpreted from a positivistic, mechanistic philosophical stance as theorems in philosophy of science. The techniques of recursive function theory are employed including ordinary and infinitary recursion theorems.

Tradeoffs in the inductive inference of nearly minimal size programs

Inductive inference machines are algorithmic devices which attempt to synthesize (in the limit) programs for a function while they examine more and more of the graph of the function. There are many possible criteria of success. We study the inference of nearly minimal size programs. Our principal results imply that nearly minimal size programs can be inferred (in the limit) without loss of inferring power provided we are willing to tolerate a finite, but not uniformly, bounded, number of anomalies in the synthesized programs. On the other hand, there is a severe reduction of inferring power in inferring nearly minimal size programs if the maximum number of anomalies allowed is any uniform constant. We obtain a general characterization for the classes of recursive functions which can be synthesized by inferring nearly minimal size programs with anomalies. We also obtain similar results for Popperian inductive inference machines. The exact tradeoffs between mind change bounds on inductive inference machines and anomalies in synthesized programs are obtained. The techniques of recursive function theory including the recursion theorem are employed.

Iterated Limiting Recursion and the Program Minimization Problem

The general problem of finding minimal programs realizing given “program descriptions” is considered, where program descriptions may be of finite or infinite length and may specify arbitrary program properties. The problem of finding minimal programs consistent with finite or infinite input-output lists is a special case (for infinite input-output lists, this is a variant of E. M. Gold's function identification problem). Although most program minimization problems are not recursively solvable, they are found to be no more difficult than the problem of deciding whether any given program realizes any given description, or the problem of enumerating programs in order of nondecreasing length (whichever is harder). This result is formulated in terms of k -limiting recursive predicates and functionals, defined by repeated application of Gold's limit operator. A simple consequence is that the program minimization problem is limiting recursively solvable for finite input-output lists and 2-limiting recursively solvable for infinite input-output lists, with weak assumptions about the measure of program size. Gold regarded limiting function identification (more generally, “black box” identification) as a model of inductive thought. Intuitively, iterated limiting identification might be regarded as higher-order inductive inference performed collectively by an ever-growing community of lower order inductive inference machines.