Abstract

Efficiently learning interpretable policies for complex tasks from demonstrations is a challenging problem. We present Hierarchical Inference with Logical Options (HILO), a novel learning algorithm that learns to imitate expert demonstrations by learning the rules that the expert is following. The rules are represented as linear temporal logic (LTL) formulas, which are interpretable and capable of encoding complex behaviors. Unlike previous works, which learn rules from high-level propositions, HILO learns rules by taking both propositions and low-level trajectories as input. It does this by defining a Bayesian model over LTL formulas, propositions, and low-level trajectories. The Bayesian model bridges the gap from formula to low-level trajectory by using a planner to find an optimal policy for a given LTL formula. Stochastic variational inference is then used to find a posterior distribution over formulas and policies given expert demonstrations. We show that by learning rules from both propositions and low-level states, HILO outperforms previous work on a rule-learning task and on four planning tasks while needing less data. We also validate HILO in the real world by teaching a robotic arm a complex packing task.

Highlights

  • I N the imitation learning (IL) problem, desired behaviors are learned by imitating expert demonstrations [1]

  • We introduced Hierarchical Inference with Logical Options (HILO), a method for inferring and planning with linear temporal logic (LTL) formulas given low-level trajectory demonstrations

  • We showed how HILO improves over other work by incorporating planning in the inference loop

Read more

Summary

INTRODUCTION

I N the imitation learning (IL) problem, desired behaviors are learned by imitating expert demonstrations [1]. Efficient manner by introducing Hierarchical Inference with Logical Options (HILO), an IL algorithm that learns a policy by learning the rules that the expert is following. Given a low-level environment including propositions; a set of pretrained low-level policies; and expert demonstrations, HILO learns a distribution over LTL formulas and policies that characterize the task the expert is performing. HILO achieves this by defining a hierarchical Bayesian model that relates LTL formulas to propositions and low-level trajectories. This paper makes the following contributions: 1) We introduce a hierarchical Bayesian model that incorporates the LOF-VI planner to relate LTL formulas to policies, thereby defining a joint distribution over LTL formulas, propositions, and low-level trajectories. 2) We use the Bayesian model to define a stochastic variational inference problem that infers a posterior distribution over interpretable LTL formulas and policies given a set of expert demonstrations. We validate HILO in a real-world setting by teaching a robotic arm a complex grocery-packing task

RELATED WORK
PRELIMINARIES
PROBLEM STATEMENT
HIERARCHICAL INFERENCE WITH LOGICAL OPTIONS
Bayesian Model
Sampling LTL formulas
EXPERIMENTS & RESULTS
Inference over Low-level States
Planning
Real-world Packing
CONCLUSION & FUTURE WORK
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call