A Deep Reinforcement Learning Approach to Learning Tree Edit Policies on Binary Trees

Abstract

Modeling sequential transformations between two trees is a fundamental task in domains such as bioinformatics. Traditional methods to address this usually rely on hand-coded heuristics and algorithms which are often hard to derive. The large advances in deep learning technologies and computational power have recently opened up new potential avenues. In particular, representation and policy learning provides an interesting opportunity to model the dynamic evolution of two trees, where each tree can be embedded in a Euclidean space and its evolution can be modeled by an embedding trajectory in this space.In this paper, we propose a representation and policy learning framework that learns a representation for arbitrary sized binary tree pairs using recurrent LSTM networks and a policy to transfer one tree in to the corresponding target tree using Reinforcement Learning. Here, the representation is pre-trained on tree transfer similarity to transform pairs of tree-structured data into an approximate numerical multidimensional vector which encodes the original structure information. This model, used with a deep reinforcement learning approach, yields a constructive method for generating basis functions for approximating value functions and permits to learn an efficient, general tree transfer policy that incrementally transforms the source tree into the target tree.