Learning Logic Programming Research Articles

Learning logic programs by explaining their failures

Scientists form hypotheses and experimentally test them. If a hypothesis fails (is refuted), scientists try to explain the failure to eliminate other hypotheses. The more precise the failure analysis the more hypotheses can be eliminated. Thus inspired, we introduce failure explanation techniques for inductive logic programming. Given a hypothesis represented as a logic program, we test it on examples. If a hypothesis fails, we explain the failure in terms of failing sub-programs. In case a positive example fails, we identify failing sub-programs at the granularity of literals. We introduce a failure explanation algorithm based on analysing branches of SLD-trees. We integrate a meta-interpreter based implementation of this algorithm with the test-stage of the Popper ILP system. We show that fine-grained failure analysis allows for learning fine-grained constraints on the hypothesis space. Our experimental results show that explaining failures can drastically reduce hypothesis space exploration and learning times.

Neurosymbolic Reasoning and Learning with Restricted Boltzmann Machines

Knowledge representation and reasoning in neural networks has been a long-standing endeavour which has attracted much attention recently. The principled integration of reasoning and learning in neural networks is a main objective of the area of neurosymbolic Artificial Intelligence. In this paper, a neurosymbolic system is introduced that can represent any propositional logic formula. A proof of equivalence is presented showing that energy minimization in restricted Boltzmann machines corresponds to logical reasoning. We demonstrate the application of our approach empirically on logical reasoning and learning from data and knowledge. Experimental results show that reasoning can be performed effectively for a class of logical formulae. Learning from data and knowledge is also evaluated in comparison with learning of logic programs using neural networks. The results show that our approach can improve on state-of-the-art neurosymbolic systems. The theorems and empirical results presented in this paper are expected to reignite the research on the use of neural networks as massively-parallel models for logical reasoning and promote the principled integration of reasoning and learning in deep networks.

Learning Logic Programs by Discovering Where Not to Search

The goal of inductive logic programming (ILP) is to search for a hypothesis that generalises training examples and background knowledge (BK). To improve performance, we introduce an approach that, before searching for a hypothesis, first discovers "where not to search". We use given BK to discover constraints on hypotheses, such as that a number cannot be both even and odd. We use the constraints to bootstrap a constraint-driven ILP system. Our experiments on multiple domains (including program synthesis and inductive general game playing) show that our approach can (i) substantially reduce learning times by up to 97%, and (ii) can scale to domains with millions of facts.

Learning Logic Programs Though Divide, Constrain, and Conquer

We introduce an inductive logic programming approach that combines classical divide-and-conquer search with modern constraint-driven search. Our anytime approach can learn optimal, recursive, and large programs and supports predicate invention. Our experiments on three domains (classification, inductive general game playing, and program synthesis) show that our approach can increase predictive accuracies and reduce learning times.

GENSYNTH: Synthesizing Datalog Programs without Language Bias

Techniques for learning logic programs from data typically rely on language bias mechanisms to restrict the hypothesis space. These methods are therefore limited by the user's ability to tune them such that the hypothesis space is simultaneously large enough to include the target program but small enough to admit a tractable search. We propose a technique to learn Datalog programs from input-output examples without requiring the user to specify any language bias. It employs an evolutionary search strategy that mutates candidate programs and evaluates their fitness on the examples using an off-the-shelf Datalog interpreter. We have implemented our approach in a tool called GenSynth and evaluate it on diverse tasks from knowledge discovery, program analysis, and relational queries. Our experiments show that GenSynth can learn correct programs from few examples, including for tasks that require recursion and invented predicates, and is robust to noise.

Learning Disjunctive Logic Programs from Nondeterministic Interpretation Transitions

Inductive logic programming (ILP) is a framework of learning logic programs from examples and background knowledge. In some real-world applications, we are particularly interested in learning aspects of system dynamics that are characterized by state transitions for which logic programs are shown to be expressive. In this work, we study this particular form of ILP—learning logic programs from state/interpretation transitions. Firstly, we define a state transition operator for disjunctive logic programs which generalizes the immediate consequence operator for normal logic programs. Secondly, we formulate two resolutions to simplify logic programs under state transition and study their properties. Finally, we put forward an inductive learning framework, which is shown to provide a sound and complete procedure for learning disjunctive logic programs from state transitions. A prototype system is implemented in Python and evaluated with randomly generated examples of well-known Boolean network benchmarks.

Learning Inference Rules from Data

This paper considers the possibility of designing AI that can learn logical or non-logical inference rules from data. We first provide an abstract framework for learning logics. In this framework, an agent $${{{\mathcal {A}}}}$$ provides training examples that consist of formulas S and their logical consequences T. Then a machine $${{{\mathcal {M}}}}$$ builds an axiomatic system that makes T a consequence of S. Alternatively, in the absence of an agent $$\mathcal{A}$$ , a machine $${{{\mathcal {M}}}}$$ seeks an unknown logic underlying given data. We next consider the problem of learning logical inference rules by induction. Given a set S of propositional formulas and their logical consequences T, the goal is to find deductive inference rules that produce T from S. We show that an induction algorithm LF1T, which learns logic programs from interpretation transitions, successfully produces deductive inference rules from input data. Finally, we consider the problem of learning non-logical inference rules. We address three case studies for learning abductive inference, frame axioms, and conversational implicature. Each case study uses machine learning techniques together with metalogic programming.

Exploiting Answer Set Programming with External Sources for Meta-Interpretive Learning

AbstractMeta-Interpretive Learning (MIL) learns logic programs from examples by instantiating meta-rules, which is implemented by the Metagol system based on Prolog. Viewing MIL-problems as combinatorial search problems, they can alternatively be solved by employing Answer Set Programming (ASP), which may result in performance gains as a result of efficient conflict propagation. However, a straightforward ASP-encoding of MIL results in a huge search space due to a lack of procedural bias and the need for grounding. To address these challenging issues, we encode MIL in the HEX-formalism, which is an extension of ASP that allows us to outsource the background knowledge, and we restrict the search space to compensate for a procedural bias in ASP. This way, the import of constants from the background knowledge can for a given type of meta-rules be limited to relevant ones. Moreover, by abstracting from term manipulations in the encoding and by exploiting the HEX interface mechanism, the import of such constants can be entirely avoided in order to mitigate the grounding bottleneck. An experimental evaluation shows promising results.

Incremental and Iterative Learning of Answer Set Programs from Mutually Distinct Examples

AbstractOver the years the Artificial Intelligence (AI) community has produced several datasets which have given the machine learning algorithms the opportunity to learn various skills across various domains. However, a subclass of these machine learning algorithms that aimed at learning logic programs, namely the Inductive Logic Programming algorithms, have often failed at the task due to the vastness of these datasets. This has impacted the usability of knowledge representation and reasoning techniques in the development of AI systems. In this research, we try to address this scalability issue for the algorithms that learn answer set programs. We present a sound and complete algorithm which takes the input in a slightly different manner and performs an efficient and more user controlled search for a solution. We show via experiments that our algorithm can learn from two popular datasets from machine learning community, namely bAbl (a question answering dataset) and MNIST (a dataset for handwritten digit recognition), which to the best of our knowledge was not previously possible. The system is publicly available athttps://goo.gl/KdWAcV.

Analogies Between Logic Programming and Linguistics For Developing Students' Understanding of Argumentation Texts

Aim/Purpose: Research shows that students encounter difficulties in identifying the structure of argumentation texts and in understanding the main message of the argument. The research examined the effect that learning Logic Programming (LP), while applying logic inference, has on students’ understanding of argumentation texts. Background: Understanding an argumentation text means exposure to its structure, which requires the ability to identify the argument presented and to distinguish between the argument and its justifications. Argumentation is an important cognitive capacity for handling conflicting information, viewpoints, and opinions. Students’ lack of ability to identify the structure of argumentation texts, and to understand its’ main message, affects the understanding of texts in general, the writing of texts, and the presentation of oral arguments. Since Logic Programming is based on inference that is similar to the way in which people commonly believe that human inferential thinking is performed, our research approach was to investigate how learning LP in Computer Science affects the understanding of argumentation texts in Linguistics. Methodology: The research population included 319 11th-grade students from five high schools, divided into a study group and a control group. Students’ understanding was tested using knowledge questionnaires after completing their language studies, before (pre-study) and after (post-study) a year of learning LP. The knowledge questionnaires included argumentation paragraphs where students were asked to give each paragraph a title and to analyze the argument structure. In addition, an attitudes questionnaire was administered at the end of the school year in order to examine the students’ attitudes towards the connection between the two disciplines. The research applied a mixed method approach, combining both qualitative and quantitative methods. Contribution: The research and its’ findings contribute to the previous body of knowledge with relation to students difficulties in understanding argumentation texts in Linguistics studies. Moreover, it suggests a new approach of using argumentation in the framework of inference as apply in LP to scaffold students’ conceptions. The use of an interactive computerized system (like the logic programming language Prolog) can scaffold students in constructing their knowledge, develop their computational thinking skills, and also enables to vary the teaching methods. Findings: Findings show that the students’ understanding of argumentation texts improved after learning LP. The study group students’ achievements were explicitly better compared with the control group students, who did not learn LP, though this was not always reflected with significant statistics. Students’ attitudes questionnaire revealed that students did not identify on their own the connections between the two disciplines and so could not explicitly use it to promote their understanding. Recommendations for Practitioners: Creative educators, who value challenges, can greatly benefit their students if they collaborate in aim for applying interdisciplinary learning while combining those two disciplines. The research conclusions shows that it is possible to improve students’ understanding if teachers explicitly mediate and guide students in drawing analogies. Recommendation for Researchers: The analysis tool we developed and apply can be used by educators and researchers to evaluate the understanding of argumentative texts by learners. It can be used in language classes at all levels as well as by educators in other disciplines in which the understanding of the argumentative structure is fundamental. Impact on Society: Developing argumentation skills and computational thinking skills. Future Research: Vary future possible research can follow the presented approach: examining how LP teachers expose the logical structure of an argumentation paragraph when they write logic programs that describe the inference represented in texts; examining how language teachers coupe with learning and using LP; examines the knowledge and skills of students that experienced a mediate learning process in the two disciplines in parallel.

Using screencasts to enhance coding skills: The case of logic programming?

Learning technologies are gradually becoming an integral part of teaching in both face-to-face and online learning. Among them, screencasts (i.e. desktop video recordings of presentations normally accompanied by the presenter?s video and narration), constitute a pedagogical tool used to create visual material to be distributed to students. Learners can then watch the videos in their own convenience and pace. The plethora of tools available makes it easier for the teachers to produce high-quality, low-cost screencasts for a number of courses. In the current paper we investigate how students perceive the impact of screencasts on their understanding and motivation in learning Logic Programming. We take the opportunity to present some tips and techniques that can be applied in any screencast production. We discuss in detail how screencasts can be used in programming courses, irrespectively of whether the latter use Imperative, Object-Oriented, or Declarative languages, and we present a number of examples to demonstrate how screencasts can facilitate learning. Furthermore, we focus particularly on Logic Programming, which lends itself to technology enhanced learning, since it requires a non-linear, out-of-the-box way of thinking towards developing programs. Finally, we evaluate our approach by presenting the opinion of students.

Teaching and learning logic programming in virtual worlds using interactive microworld representations

AbstractLogic Programming (LP) follows the declarative programming paradigm, which novice students often find hard to grasp. The limited availability of visual teaching aids for LP can lead to low motivation for learning. In this paper, we present a platform for teaching and learning Prolog in Virtual Worlds, which enables the visual interpretation and verification of program results in a straightforward fashion and requires students to adopt a collaborative problem‐solving approach. The results of the pilot application and student‐centered evaluation of the platform are encouraging regarding group learning performance and user experience. The paper contributes to current practice of teaching and learning LP by proposing a metaphor and a system that can empower the educational process with toy world examples visualized in a shared 3D environment.

Learning concepts and their unions from positive data with refinement operators

This paper is concerned with a sufficient condition under which a concept class is learnable in Gold's classical model of identification in the limit from positive data. The standard principle of learning algorithms working under this model is called the MINL strategy, which is to conjecture a hypothesis representing a minimal concept among the ones consistent with the given positive data. The minimality of a concept is defined with respect to the set-inclusion relation --- the strategy is semantics-based. On the other hand, refinement operators have been developed in the field of learning logic programs, where a learner constructs logic programs as hypotheses consistent with given logical formulae. Refinement operators have syntax-based definitions --- they are defined based on inference rules in first-order logic. This paper investigates the relation between the MINL strategy and refinement operators in inductive inference. We first show that if a hypothesis space admits a refinement operator with certain properties, the concept class will be learnable by an algorithm based on the MINL strategy. We then present an additional condition that ensures the learnability of the class of unbounded finite unions of concepts. Furthermore, we show that under certain assumptions a learning algorithm runs in polynomial time.

A Survey on Inductive Logic Programming Learning Algorithm

A Survey on Inductive Logic Programming Learning Algorithm

Learning Logic Programming in Radial Basis Function Network via Genetic Algorithm

Learning Logic Programming in Radial Basis Function Network via Genetic Algorithm

Improving high-level and gate-level testing with FATE: A functional automatic test pattern generator traversing unstabilised extended FSM

A functional automatic test pattern generator (ATPG) that explores the design under test (DUT) state space by exploiting an easy-to-traverse extended finite state machine (FSM) model has been described. The ATPG engine relies on learning, backjumping and constraint logic programming to deterministically generate test vectors for traversing all transitions of the extended FSM. Testing of hard-to-detect faults is thus improved. The generated test sequences are very effective in detecting both high-level faults and gate-level stuck-at faults. Thus, the reuse of test sequences generated by the proposed ATPG allows also to improve the stuck-at fault coverage and to reduce the execution time of commercial gate-level ATPGs.

Mind change complexity of learning logic programs

The present paper motivates the study of mind change complexity for learning minimal models of length-bounded logic programs. It establishes ordinal mind change complexity bounds for learnability of these classes both from positive facts and from positive and negative facts. Building on Angluin's notion of finite thickness and Wright's work on finite elasticity, Shinohara defined the property of bounded finite thickness to give a sufficient condition for learnability of indexed families of computable languages from positive data. This paper shows that an effective version of Shinohara's notion of bounded finite thickness gives sufficient conditions for learnability with ordinal mind change bound, both in the context of learnability from positive data and for learnability from complete (both positive and negative) data. Let ω be a notation for the first limit ordinal. Then, it is shown that if a language defining framework yields a uniformly decidable family of languages and has effective bounded finite thickness, then for each natural number m>0, the class of languages defined by formal systems of length ⩽ m: • is identifiable in the limit from positive data with a mind change bound of ω m ; • is identifiable in the limit from both positive and negative data with an ordinal mind change bound of ω× m. The above sufficient conditions are employed to give an ordinal mind change bound for learnability of minimal models of various classes of length-bounded Prolog programs, including Shapiro's linear programs, Arimura and Shinohara's depth-bounded linearly covering programs, and Krishna Rao's depth-bounded linearly moded programs. It is also noted that the bound for learning from positive data is tight for the example classes considered.

Learning logic programs with structured background knowledge

The efficient learnability of restricted classes of logic programs is studied in the PAC framework of computational learning theory. We develop the product homomorphism method, which gives polynomial PAC learning algorithms for a nonrecursive Horn clause with function-free ground background knowledge, if the background knowledge satisfies some structural properties. The method is based on a characterization of the concept that corresponds to the relative least general generalization of a set of positive examples with respect to the background knowledge. The characterization is formulated in terms of products and homomorphisms. In the applications this characterization is turned into an explicit combinatorial description, which is then translated into the language of nonrecursive Horn clauses. We show that a nonrecursive Horn clause is polynomially PAC-learnable if there is a single binary background predicate and the ground atoms in the background knowledge form a forest. If the ground atoms in the background knowledge form a disjoint union of cycles then the situation is different, as the shortest consistent hypothesis may have exponential size. In this case polynomial PAC-learnability holds if a different representation language is used. We also consider the complexity of hypothesis finding for multiple clauses in some restricted cases.

Relational Reinforcement Learning

Relational reinforcement learning is presented, a learning technique that combines reinforcement learning with relational learning or inductive logic programming. Due to the use of a more expressiv...

Grammatically biased learning: Learning logic programs using an explicit antecedent description language

Every concept learning system produces hypotheses that are written in some sort of constrained language called the concept description language, and for most learning systems, the concept description language is fixed. This paper describes a learning system that makes a large part of the concept description language an explicit input, and discusses some of the possible applications of providing this additional input. In particular, we discuss a technique for learning a logic program such that the antecedent of each clause in the program can be generated by a special antecedent description language; it is shown that this technique can be used to make use of many different types of background knowledge, including constraints on how predicates can be used, programming clichés, overgeneral theories, incomplete theories, and theories syntactically close to the target theory. The approach thus unifies many of the problems previously studied in the field of knowledge-based learning.