Solving a Spatial Puzzle Using Answer Set Programming Integrated with Markov Decision Process

Abstract

Spatial puzzles are interesting domains to investigate problem solving, since the reasoning processes involved in reasoning about spatial knowledge is one of the essential items for an agent to interact in the human environment. With this in mind, the goal of this work is to investigate the knowledge representation and reasoning process related to the solution of a spatial puzzle, the Fisherman's Folly, composed of flexible string, rigid objects and holes. To achieve this goal, the present paper uses heuristics (obtained after solving a relaxed version of the puzzle) to accelerate the learning process, while applying a method that combines Answer Set programming (ASP) with Reinforcement learning (RL), the oASP(MDP) algorithm, to find a solution to the puzzle. ASP is the logic language chosen to build the set of states and actions of a Markov Decision Process (MDP) representing the domain, where RL is used to learn the optimal policy of the problem.