Deep Reinforcement Learning Guided Decision Tree Learning For Program Synthesis

Abstract

Syntax-Guided Synthesis (SyGuS) is a general solving framework for program synthesis. Previous researchers have proposed a divide-and-conquer strategy to divide the grammar search space in order to search separately. This framework converts the program synthesis problem into a decision tree classification problem. It unifies the obtained partial results into a complete program by learning decision trees according to conditional expression statements. However, due to the unknowns and uncertainties of sample label categories and sample attribute features in program synthesis, as well as the limitation of traditional decision tree learning methods to immediate gains, it results in long solving times and the large size of the candidate program. To address the above problems, we propose a deep reinforcement learning-based synthesis strategy for syntax-guided programs: a) the decision tree learning process is modeled as a Markov chain decision process, addressing the learning of knowledge patterns under conceptual drift and the weighting of the importance of sample attribute features; b) a graph neural network is used to extract the feature information of the sample attributes as action embedding feature values; (c) a reward function is proposed to evaluate the classification accuracy of the sample attributes as a feedback mechanism. We have implemented our approach in a tool called RLSolver. According to the experiments, the solving times ofRLSolver are comparable to EUSolver on small-scale task datasets. RLSolver substantially reduces the solving times on large-scale task datasets, solving two more SyGus tasks than EUSolver in the same solving time limit. In addition, RLSolver reduces the number of decision tree learning rounds by 60% and the size of decision tree size by 40% after using the multi-round model training strategy and early-stop strategy.