CONSTRUCTIVE INDUCTION OF GOAL CONCEPTS FROM AGENT PERCEPTS AND REINFORCEMENT FEEDBACK

Abstract

This paper investigates the ability of an agent to recognize unseen goal states in an observable gridworld environment. This ability is important in order to allow the explicit planning of agent’s action sequences. Gridworld states are described by the collection of attributes instead of “atomic” labels. This attribute-based state representation allows generalizing over the attributes and discriminating goal (reinforced) states from non-goal states. We argue that a biased induction technique is required to solve this supervised learning problem. A constructive induction technique is proposed which is able to discover goal concepts relevant to the grid-world environments, where training instances are 2D image-like objects. The set of constructive operators is defined and a search procedure in the space of sequences of these operators is described. The proposed technique was tested on two non-trivial sample tasks and was successful in learning goal concepts. It allowed synonymous descriptions of the same goal concept to be learnt depending on the set of operators and their activation order. The complete assessment of prospects and limits of the proposed constructive induction technique still needs to be done.