SLUG: A connectionist architecture for inferring the structure of finite-state environments

Abstract

Consider a robot wandering around an unfamiliar environment, performing actions and sensing the resulting environmental states. The robot's task is to construct an internal model of its environment, a model that will allow it to predict the consequences of its actions and to determine what sequences of actions to take to reach particular goal states. Rivest and Schapire (1987a, 1987bs Schapire, 1988) have studied this problem and have designed a symbolic algorithm to strategically explore and infer the structure of “finite state” environments. The heart of this algorithm is a clever representation of the environment called an update graph. We have developed a connectionist implementation of the update graph using a highly-specialized network architecture. With back propagation learning and a trivial exploration strategy—choosing random actions—the network can outperform the Rivest and Schapire algorithm on simple problems. Perhaps the most interesting consequence of the connectionist approach is that, by relaxing the constraints imposed by a symbolic description, it suggests a more general representation of the update graph, thus allowing for greater flexibility in expressing potential solutions.