Application Of Inductive Logic Programming Research Articles

Stephen H. Muggleton and Hiroaki Watanabe (Eds.): Latest advances in inductive logic programming

Inductive logic programming (ILP) investigates the construction of logic programs from training examples and background knowledge, where a logic program is a set of sentences in logical form, expressing facts and rules about some domain. ILP is more powerful than traditional machine learning methods, such as C4.5 decision trees, because it uses an expressive first-order logic framework and readily allows the user to incorporate background knowledge. In first-order logic, sentences are formed by using constants, variables, predicates, functions, quantifiers, and connectives in a well-formed format, where constants are the objects in the domain of discourse, variables range over the objects in the domain, predicates specify the properties of objects or the relations among objects, functions map object(s) onto other object(s), quantifiers are 8 (Universal) and 9 (Existential) that quantify the variables, and connectives include negation, and, or, etc. ILP has a strong theoretical foundation from logic programming and computational learning theory. There has been significant progress in ILP in the last two decades and this book contains 25 papers associated with the 21st International Conference of Inductive Logic Programming which was held in Cumberland Lodge, an educational charity and a unique conference center in the heart of the Great Park, Windsor, United Kingdom. The book is divided into 7 parts and the first part has 9 papers about applications of ILP in different domains. In Part 2 five research studies that extend ILP to probabilistic logical learning are presented. Part 3 contains a number of papers about the implementations of ILP systems. Research studies about theory, logical learning, constraints, as well as spatial and temporal learning are presented in parts 4–7.

Extraction of Social Structure Change from Survey Data Application of Inductive Logic Programming for cohort network analysis

Most existing works on network analysis mainly focus on only the existence of relations between entities. However, in trying to understand a real network, we naturally use not only the existence of relations but also information on the kind of relations, the attributes in the nodes, and the changes in time. In addition, we can observe some of the measures that are obtained as a result of the whole network structure. In order to extract some meaningful structural changes and integrity constraints from a dynamical network constructed from survey data, we are proposing a novel data mining framework in this paper that includes the above information that has not been used in previous studies. In the proposed framework, we start by detecting the change point in the dynamic network according to the change in the characteristic quantity. Then, by using the detected points, a dynamic network will be divided into two groups. In other words, we associate the class information to each network in a dynamic network. Finally, meaningful structural changes and integrity constraints can be obtained by applying inductive logic programming to a dynamic network and the related background knowledge represented in the first order logic. In experiments using real world data, we succeeded in obtaining meaningful results. Thus, we confirmed the usefulness of the proposed framework.

Inductive Logic Programming and Embodied Agents

Open-ended learning is regarded as the ultimate milestone, especially in intelligent robotics. Preferably it should be unsupervised and it is by its nature inductive. In this article we want to give an overview of attempts to use Inductive Logic Programming (ILP) as a machine learning technique in the context of embodied autonomous agents. Relatively few such attempts exist altogether and the main goal in reviewing several of them was to find a thorough understanding of the difficulties that the application of ILP has in general and especially in this area. The second goal was to review any possible directions for overcoming these obstacles standing on the way of more widespread use of ILP in this context of embodied autonomous agents. Whilst the most serious problems, the mismatch between ILP and the large datasets encountered with embodied autonomous agents seem difficult to overcome we also found interesting research actively pursuing to alleviate these problems.

Mathematical applications of inductive logic programming

The application of Inductive Logic Programming to scientific datasets has been highly successful. Such applications have led to breakthroughs in the domain of interest and have driven the development of ILP systems. The application of AI techniques to mathematical discovery tasks, however, has largely involved computer algebra systems and theorem provers rather than machine learning systems. We discuss here the application of the HR and Progol machine learning programs to discovery tasks in mathematics. While Progol is an established ILP system, HR has historically not been described as an ILP system. However, many applications of HR have required the production of first order hypotheses given data expressed in a Prolog-style manner, and the core functionality of HR can be expressed in ILP terminology. In Colton (2003), we presented the first partial description of HR as an ILP system, and we build on this work to provide a full description here. HR performs a novel ILP routine called Automated Theory Formation, which combines inductive and deductive reasoning to form clausal theories consisting of classification rules and association rules. HR generates definitions using a set of production rules, interprets the definitions as classification rules, then uses the success sets of the definitions to induce hypotheses from which it extracts association rules. It uses third party theorem provers and model generators to check whether the association rules are entailed by a set of user supplied axioms.HR has been applied successfully to a number of predictive, descriptive and subgroup discovery tasks in domains of pure mathematics. We survey various applications of HR which have led to it producing number theory results worthy of journal publication, graph theory results rivalling those of the highly successful Graffiti program and algebraic results leading to novel classification theorems. To further promote mathematics as a challenge domain for ILP systems, we present the first application of Progol to an algebraic domain—we use Progol to find algebraic properties of quasigroups, semigroups and magmas (groupoids) of varying sizes which differentiate pairs of non-isomorphic objects. This development is particularly interesting because algebraic domains have been an important proving ground for both deduction systems and constraint solvers. We believe that AI programs written for discovery tasks will need to simultaneously employ a variety of reasoning techniques such as induction, abduction, deduction, calculation and invention. We argue that mathematics is not only a challenging domain for the application of ILP systems, but that mathematics could be a good domain in which to develop a new generation of systems which integrate various reasoning techniques.

Compact Representation of Knowledge Bases in Inductive Logic Programming

In many applications of Inductive Logic Programming (ILP), learning occurs from a knowledge base that contains a large number of examples. Storing such a knowledge base may consume a lot of memory. Often, there is a substantial overlap of information between different examples. To reduce memory consumption, we propose a method to represent a knowledge base more compactly. We achieve this by introducing a meta-theory able to build new theories out of other (smaller) theories. In this way, the information associated with an example can be built from the information associated with one or more other examples and redundant storage of shared information is avoided. We also discuss algorithms to construct the information associated with example theories and report on a number of experiments evaluating our method in different problem domains.

EXPERIMENTS IN PREDICTING BIODEGRADABILITY

This paper is concerned with the use of AI techniques in ecology. More specifically, we present a novel application of inductive logic programming (ILP) in the area of quantitative structure-activity relationships (QSARs). The activity we want to predict is the biodegradability of chemical compounds in water. In particular, the target variable is the half-life for aerobic aqueous biodegradation. Structural descriptions of chemicals in terms of atoms and bonds are derived from the chemicals' SMILES encodings. The definition of substructures is used as background knowledge. Predicting biodegradability is essentially a regression problem, but we also consider a discretized version of the target variable. We thus employ a number of relational classification and regression methods on the relational representation and compare these to propositional methods applied to different propositionalizations of the problem. We also experiment with a prediction technique that consists of merging upper and lower bound predictions into one prediction. Some conclusions are drawn concerning the applicability of machine learning systems and the merging technique in this domain and the evaluation of hypotheses.

Structure Activity Relationships (SAR) and Pharmacophore Discovery Using Inductive Logic Programming (ILP)

AbstractThe application of Inductive Logic Programming (ILP), a form of machine learning, to derive structure activity relationships (SAR) and to discover pharmacophores is reported. The ILP approach was initially applied to model 1D SARs in terms of the attributes of the molecules. Subsequently 2D ILP SARs were developed describing chemical connectivity. Finally ILP has been used to model 3D SARs in which the conformation of the pharmacophore can be described. ILP has advantages over many other widely used methods as it can reason with relations and hence discover chemical substructures and 3D features without these aspects having been explicitly encoded prior to learning. In particular, there is no requirement for a structural superposition. Additionally, the results of ILP provide chemical descriptions that can readily be understood by a medicinal chemist. In several trials, ILP‐based SARs have been shown to be significantly more accurate than widely‐used methods.

日本語翻訳タスクへの帰納論理プログラミングの適用

日本語翻訳タスクへの帰納論理プログラミングの適用

Inductive Logic Programming: Issues, results and the challenge of Learning Language in Logic

Inductive Logic Programming (ILP) is the area of AI which deals with the induction of hypothesised predicate definitions from examples and background knowledge. Logic programs are used as a single representation for examples, background knowledge and hypotheses. ILP is differentiated from most other forms of Machine Learning (ML) both by its use of an expressive representation language and its ability to make use of logically encoded background knowledge. This has allowed successful applications of ILP in areas such as molecular biology and natural language which both have rich sources of background knowledge and both benefit from the use of an expressive concept representation languages. For instance, the ILP system Progol has recently been used to generate comprehensible descriptions of the 23 most populated fold classes of proteins, where no such descriptions had previously been formulated manually. In the natural language area ILP has not only been shown to have higher accuracies than various other ML approaches in learning the past tense of English but also shown to be capable of learning accurate grammars which translate sentences into deductive database queries. The area of Learning Language in Logic (LLL) is producing a number of challenges to existing ILP theory and implementations. In particular, language applications of ILP require revision and extension of a hierarchically defined set of predicates in which the examples are typically only provided for predicates at the top of the hierarchy. New predicates often need to be invented, and complex recursion is usually involved. Advances in ILP theory and implementation related to the challenges of LLL are already producing beneficial advances in other sequence-oriented applications of ILP. In addition LLL is starting to develop its own character as a sub-discipline of AI involving the confluence of computational linguistics, machine learning and logic programming.

Relational knowledge discovery in a chinese character database

This article describes a novel application ofInductive Logic Programming (ILP) to the problem of data mining relational databases. The task addressed here consists in mining a relational database ofmore than 200,000 ground facts describing 6768 Chinese characters. Mining this relational database may be recast in an ILP setting, where the form ofthe association rules searched are represented as nondeterminate Horn clauses, a type of clause known for being computationally hard to learn. We have introduced a new kind of language bias, S-structural indeterminate clauses, which takes into account the meaning of part-of predicates that play a key role in the complexity oflearning in structural domains. The ILP algorithm REPART has been specifically developed to learn S-structural indeterminate clauses. Its efficiency lies in a particular change ofrepresentation, so as to enable one to use propositional learners. This article presents original results discovered by REPART that exemplify how ILP algorithms may not only scale up efficiently to large relational databases but also discover useful and computationally hard to learn patterns.

Diterpene structure elucidation from 13cnmr spectra with inductive logic programming

We present a novel application ofInductive Logic Programming (ILP) to the problem of diterpene structure elucidation from 13 CNMR spectra. Diterpenes are organic compounds oflow molecular weight with a skeleton of 20 carbon atoms. They are of significant chemical and commercial interest because oftheir use as lead compounds in the search for new pharmaceutical effectors. The interpretation of diterpene 13 CNMR spectra normally requires specialists with detailed spectroscopic knowledge and substantial experience in natural products chemistry, specifically knowledge on peak patterns and chemical structures. Given a database ofpeak patterns for diterpenes with known structure, we apply several ILP approaches to discover correlations between peak patterns and chemical structure. The approaches used include first - order inductive learning, relational instance based learning, induction oflogical decision trees, and inductive constraint logic. Performance close to that of domain experts is achieved, which suffices for practical use.

Using inductive logic programming to discover knowledge hidden in chemical data

This paper demonstrates how general purpose tools from the field of inductive logic programming (ILP) can be applied to analytical chemistry. As far as these authors are aware, this is the first published work to describe the application of the ILP tool Golem to separation science. An outline of the theory of ILP is given, together with a description of Golem and previous applications of ILP. The advantages of ILP over classical machine induction techniques, such as the Top-Down-Induction-of-Decision-Tree family, are explained. A case-study is then presented in which Golem is used to induce rules which predict, with a high accuracy (82%), whether each of a series of attempted separations succeed or fail. The separation data was obtained from published work on the attempted separation of a series of 3-substituted phthalide enantiomer pairs on (R)- N-(3,5-dinitrobenzoyl)phenylglycine.

Engineering applications of ILP

Several applications of Inductive Logic Programming (ILP) are presented. These belong to various areas of engineering, including mechanical, environmental, software, and dynamical systems engineering. The particular applications are finite element mesh design, biological classification of river water quality, data reification, inducing program invariants, learning qualitative models of dynamic systems, and learning control rules for dynamic systems. A number of other applications are briefly mentioned. Finally, a discussion of the advantages and disadvantages of ILP as compared to other approaches to machine learning is given.

Applications of inductive logic programming

Techniques of machine learning have been successfully applied to various problems [1, 12]. Most of these applications rely on attribute-based learning, exemplified by the induction of decision trees as in the program C4.5 [20]. Broadly speaking, attribute-based learning also includes such approaches to learning as neural networks and nearest neighbor techniques. The advantages of attribute-based learning are: relative simplicity, efficiency, and existence of effective techniques for handling noisy data. However, attribute-based learning is limited to non-relational descriptions of objects in the sense that the learned descriptions do not specify relations among the objects' parts. Attribute-based learning thus has two strong limitations: the background knowledge can be expressed in rather limited form, and the lack of relations makes the concept description language inappropriate for some domains.

Applications of inductive logic programming

Some applications of Inductive Logic Programming (ILP) are presented. Those applications are chosen that specifically benefit from relational descriptions generated by ILP programs, and from ILP's ability to accommodate background knowledge. Applications included are: drug design, predicting the secondary structure of proteins, and design of finite-element meshes. Some other applications are briefly described. The practical advantages and disadvantages of ILP learning are discussed.

A knowledge base for finite element mesh design

The finite element method (FEM) is the most successful numerical method, that is used extensively by engineers to analyse stresses and deformations in physical structures. These structures should be represented as a finite element mesh. Defining an appropriate geometric mesh model that ensures low approximation errors and avoids unnecessary computational overheads is a very difficult and time consuming task. It is the major bottleneck in the FEM analysis process. The inductive logic programming system GOLEM has been employed to construct the rules for deciding about the appropriate mesh resolution. Five cylindrical mesh models have been used as a source of training examples. The evaluation of the resulting knowledge base shows that conditions in the domain are well represented by the rules, which specify the required number of the finite elements on the edges of the structures to be analysed using FEM. A comparison between the results obtained by this knowledge base and conventional mesh generation techniques confirms that the application of inductive logic programming is an effective approach to solving the problem of mesh design.