Clustering subspace generalization to obtain faster reinforcement learning

Abstract

In reinforcement learning, very low and even lack of spatial generalization capability result in slow learning. Exploiting possible experience generalization in subspaces, a sub-dimension of the original state representation, is one approach to speed up learning in terms of required interactions. The target of this paper is to alleviate the detriment of the perceptual aliasing in the subspaces further to enhance the benefit of their generalization. We augment an extra dimension to the subspaces coming from a clustering process on the state-space. Through this framework, called Clustered-Model Based Learning with Subspaces (C-MoBLeS), states with similar policies are categorized to the same cluster and the agent can exploit the generalization of the subspace learning more by this localization. Therefore, the agent uses generalization of the subspaces which are in the cluster of the agent’s state. Several experiments show that C-MoBLeS increases the learning speed effectively.