Inductive Logic Programming Method Research Articles

A differentiable first-order rule learner for inductive logic programming

Learning first-order logic programs from relational facts yields intuitive insights into the data. Inductive logic programming (ILP) models are effective in learning first-order logic programs from observed relational data. Symbolic ILP models support rule learning in a data-efficient manner. However, symbolic ILP models are not robust to learn from noisy data. Neuro-symbolic ILP models utilize neural networks to learn logic programs in a differentiable manner which improves the robustness of ILP models. However, most neuro-symbolic methods need a strong language bias to learn logic programs, which reduces the usability and flexibility of ILP models and limits the logic program formats. In addition, most neuro-symbolic ILP methods cannot learn logic programs effectively from both small-size datasets and large-size datasets such as knowledge graphs. In the paper, we introduce a novel differentiable ILP model called differentiable first-order rule learner (DFORL), which is scalable to learn rules from both smaller and larger datasets. Besides, DFORL only needs the number of variables in the learned logic programs as input. Hence, DFORL is easy to use and does not need a strong language bias. We demonstrate that DFORL can perform well on several standard ILP datasets, knowledge graphs, and probabilistic relation facts and outperform several well-known differentiable ILP models. Experimental results indicate that DFORL is a precise, robust, scalable, and computationally cheap differentiable ILP model.

A logic programming approach to predict effective compiler settings for embedded software

AbstractThis paper introduces a new logic-based method for optimising the selection of compiler flags on embedded architectures. In particular, we use Inductive Logic Programming (ILP) to learn logical rules that relate effective compiler flags to specific program features. Unlike earlier work, we aim to infer human-readable rules and we seek to develop a relational first-order approach which automatically discovers relevant features rather than relying on a vector of predetermined attributes. To this end we generated a data set by measuring execution times of 60 benchmarks on an embedded system development board and we developed an ILP prototype which outperforms the current state-of-the-art learning approach in 34 of the 60 benchmarks. Finally, we combined the strengths of the current state of the art and our ILP method in a hybrid approach which reduced execution times by an average of 8% and up to 50% in some cases.

QuickFOIL

Inductive Logic Programming (ILP) is a classic machine learning technique that learns first-order rules from relational-structured data. However, to-date most ILP systems can only be applied to small datasets (tens of thousands of examples). A long-standing challenge in the field is to scale ILP methods to larger data sets. This paper presents a method called QuickFOIL that addresses this limitation. QuickFOIL employs a new scoring function and a novel pruning strategy that enables the algorithm to find high-quality rules. QuickFOIL can also be implemented as an in-RDBMS algorithm. Such an implementation presents a host of query processing and optimization challenges that we address in this paper. Our empirical evaluation shows that QuickFOIL can scale to large datasets consisting of hundreds of millions tuples, and is often more than order of magnitude more efficient than other existing approaches.

Inverse subsumption for complete explanatory induction

Modern explanatory inductive logic programming methods like Progol, Residue procedure, CF-induction, HAIL and Imparo use the principle of inverse entailment (IE). Those IE-based methods commonly compute a hypothesis in two steps: by first constructing an intermediate theory and next by generalizing its negation into the hypothesis with the inverse of the entailment relation. Inverse entailment ensures the completeness of generalization. On the other hand, it imposes many non-deterministic generalization operators that cause the search space to be very large. For this reason, most of those methods use the inverse relation of subsumption, instead of entailment. However, it is not clear how this logical reduction affects the completeness of generalization. In this paper, we investigate whether or not inverse subsumption can be embedded in a complete induction procedure; and if it can, how it is to be realized. Our main result is a new form of inverse subsumption that ensures the completeness of generalization. Consequently, inverse entailment can be reduced to inverse subsumption without losing the completeness for finding hypotheses in explanatory induction.

Using abduction and induction for operational requirements elaboration

Requirements Engineering involves the elicitation of high-level stakeholder goals and their refinement into operational system requirements. A key difficulty is that stakeholders typically convey their goals indirectly through intuitive narrative-style scenarios of desirable and undesirable system behaviour, whereas goal refinement methods usually require goals to be expressed declaratively using, for instance, a temporal logic. In actual software engineering practice, the extraction of formal requirements from scenario-based descriptions is a tedious and error-prone process that would benefit from automated tool support. This paper presents an Inductive Logic Programming method for inferring operational requirements from a set of example scenarios and an initial but incomplete requirements specification. The approach is based on translating the specification and the scenarios into an event-based logic programming formalism and using a non-monotonic reasoning system, called eXtended Hybrid Abductive Inductive Learning, to automatically infer a set of event pre-conditions and trigger-conditions that cover all desirable scenarios and reject all undesirable ones. This learning task is a novel application of logic programming to requirements engineering that also demonstrates the utility of non-monotonic learning capturing pre-conditions and trigger-conditions.

RELATIONAL DATA MINING AND ILP FOR DOCUMENT IMAGE UNDERSTANDING

Document image understanding denotes the recognition of semantically relevant components in the layout extracted from a document image. This recognition process is based on domain-specific knowledge that can be acquired automatically by applying data mining techniques. The spatial dimension of page layout makes classification methods developed in inductive logic programming (ILP) and multi-relational data mining (MRDM) the most suitable candidates for this specific task. In this paper, both approaches are considered and empirically compared on three different data sets consisting of multi-page articles published in an international journal and historical documents. The ILP method is able to learn recursive logical theories that express dependencies between logical components, while the MRDM method extends the naïve Bayesian classifier to data stored in multiple tables of a relational database. Experimental results confirm the importance of the spatial dimension for this application and show that the ILP method tends to be conservative with a high (low) percentage of omission (commission) errors, while the probabilistic nature of the MRDM method allows us to tradeoff between the two types of error.

Learning relational rule from examples that are neither positive nor negative

AbstractOver the past decade, several inductive logic programming (ILP) systems have been developed. However, the normal ILP system does not have enough power to induce logic programs in some domains. Therefore, various new ILP systems, such as nonmonotonic ILP, that apply new learning tasks, have been proposed. In the present paper, a new learning task, called relational learning from nondichotomizable examples, is proposed for relational domains. The scheme of this task is similar to that of normal ILP, with the exception of the training examples. Normally, conventional ILP uses positive and negative examples in the training process. However, the examples of the proposed learning task are not strictly positive or negative: the training examples have a continuous scale. This new learning task is defined and a new method for learning this task is proposed. A new ILP system, SYNGIP (for SYNthesized system using Genetic programming and Inductive logic Programming), is developed based on this method and the performance of SYNGIP is compared experimentally to that of the conventional ILP method for the task of human feeling acquisition. © 2001 Scripta Technica, Syst Comp Jpn, 32(14): 34–40, 2001

Logic Program Synthesis as Problem Reduction Using Combining Forms

This paper presents an approach to inductive synthesis of logic programs from examples using problem decomposition and problem reduction principles. This is in contrast to the prevailing logic program induction paradigm, which relies on generalization of programs from examples. The problem reduction is accomplished as a constrained top-down search process, which eventually is to reach trivial problems. Our induction scheme applies a distinguished logic programming language in which programs are combined from elementary predicates by means of combinators conceived of as problem reduction operators including list recursion forms. The operator form admits inductive synthesis as a top-down piecewise composition of semantically meaningful program elements according to the compositional semantics principle and with appeals neither to special generalization mechanisms nor to alternative forms of resolution and unification, or predicate invention. The search space is reduced by subjecting the induction process to various constraints concerning syntactical form, modes, data types, and computational resources. This is illustrated in the paper with well-modedness constraints with the aim of synthesising well-moded, procedurally acceptable programs. Preliminary experiments with the proposed induction method lead us to tentatively conclude that the presented approach forms a viable alternative to the prevailing inductive logic programming methods applying generalization from examples.

Inductive specification recovery: Understanding software by learning from example behaviors

We describe a technique for extracting specifications from software using machine learning techniques. In our proposed technique, instrumented code is run on a number of representative test cases, generating examples of its behavior. Inductive learning techniques are then used to generalize these examples, forming a general description of some aspect of the system's behavior. A case study is presented in which this “inductive specification recovery” method is used to find Datalog specifications forC code that implements database views, in the context of a large real-world software system. It is demonstrated that off-the-shelf inductive logic programming methods can be successfully used for specification recovery in this domain, but that these methods can be substantially improved by adapting them more closely to the task at hand.

Integrating ILP and EBL

This paper presents a review of recent work that integrates methods from Inductive Logic Programming (ILP) and Explanation-Based Learning (EBL). ILP and EBL methods have complementary strengths and weaknesses and a number of recent projects have effectively combined them into systems with better performance than either of the individual approaches. In particular, integrated systems have been developed for guiding induction with prior knowledge (ML-Smart, FOCL, GRENDEL) refining imperfect domain theories (FORTE, AUDREY, Rx), and learning effective search-control knowledge (AxA-EBL, DOLPHIN).