Efficient learning by symbol emergence in multilayer network and agent collaboration

Abstract

Two simple models, Model-A and Model-B, are described as the first step toward fusing computational and symbolic processing. Model-B is an extension of the Model-A. Each of them consists of two layers. The lower layer is a neural network in which Q-learning is simulated. The inputs are the state variables, and the outputs are the Q-values for each action. It is tuned to update and store the Q-table. The upper layer watches the activity in the lower layer to identify the group of nodes that are activated when some action in the lower layer obtains a high reward from the environment. In this way, new symbols emerge that are embedded in the lower layer to speed up the learning. Model-B is extended so as to learn more complex concepts quickly. When an important concept is learned, the corresponding symbol is generalized and embedded in a different place at a lower level. Simulation demonstrated that symbol emergence and the forced application of these symbols in Q-learning greatly improves the performance of players playing a simple football game.