Symbol emergence and its forced application in Q-learning toward fusion of computational and symbolic processing

Abstract

A simple model is described as the first step toward fusing computational and symbolic It consists of two layers. The lower layer is a neural network in which Q-learning is simulated. The inputs are the state variables, and the outputs are the Q-values for each action. It is tuned to update and store the Q-table. The upper layer watches the activity in the lower layer to identify the group of nodes that are activated when some action in the lower layer obtains a high reward from the environment. In this way, new symbols emerge that are embedded in the lower layer to speed up the learning. When an important concept is learned, the corresponding symbol is generalized and embedded in a different place at a lower level. Simulation has demonstrated that symbol emergence and the forced application of these symbols in Q-learning greatly improves the performance of players playing a simple football game. This approach is a first step towards the deep fusion of computational and symbolic processing.