Learning higher-order logic programs

Andrew Cropper,Rolf Morel,Stephen Muggleton

doi:10.1007/s10994-019-05862-7

Andrew Cropper, Rolf Morel + Show 1 more

Open Access

https://doi.org/10.1007/s10994-019-05862-7

Copy DOI

Journal: Machine Learning	Publication Date: Dec 3, 2019
Citations: 14	License type: open-access

Affiliation: University of Oxford, Imperial College London

Abstract

A key feature of inductive logic programming is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce techniques to learn higher-order programs. Specifically, we extend meta-interpretive learning (MIL) to support learning higher-order programs by allowing for higher-order definitions to be used as background knowledge. Our theoretical results show that learning higher-order programs, rather than first-order programs, can reduce the textual complexity required to express programs, which in turn reduces the size of the hypothesis space and sample complexity. We implement our idea in two new MIL systems: the Prolog system text {Metagol}_{ho} and the ASP system text {HEXMIL}_{ho}. Both systems support learning higher-order programs and higher-order predicate invention, such as inventing functions for map/3 and conditions for filter/3. We conduct experiments on four domains (robot strategies, chess playing, list transformations, and string decryption) that compare learning first-order and higher-order programs. Our experimental results support our theoretical claims and show that, compared to learning first-order programs, learning higher-order programs can significantly improve predictive accuracies and reduce learning times.

Highlights

Suppose you have intercepted encrypted messages and you want to learn a general decryption program from them
We support our claim by showing that learning higherorder programs can reduce the textual complexity required to express programs, which in turn reduces the size of the hypothesis space and sample complexity
– We show that the answer set programming (ASP)-based HEXMIL and HEXMILho have an additional factor determining the size of their search space, namely the number of constants (Sect. 4.5)

Summary

Introduction

Suppose you have intercepted encrypted messages and you want to learn a general decryption program from them. In this scenario the underlying encryption algorithm is a simple Caesar cipher with a shift of +1. Given these examples, most inductive logic programming (ILP) approaches, such as meta-interpretive learning (MIL) (Muggleton et al 2014, 2015), would learn a recur-. We extend MIL to support learning higher-order programs that use higher-order constructs such as map/3, until/4, and ifthenelse/5 Using this new approach, we can learn an equivalent yet smaller decryption program, such as the one shown, which uses map/3 to abstract away the recursion and list manipulation. We support our claim by showing that learning higherorder programs can reduce the textual complexity required to express programs, which in turn reduces the size of the hypothesis space and sample complexity

Methods

Results

Discussion

Conclusion