Developmental Network-2: The Autonomous Generation of Optimal Internal-Representation Hierarchy.

Abstract

It is very challenging for machine learning methods to reach the goal of general-purpose learning since there are so many complicated situations in different tasks. The learning methods need to generate flexible internal representations for all scenarios met before. The hierarchical internal representation is considered as an efficient way to build such flexible representations. By hierarchy, we mean important local features in the input can be combined to form higher level features with more context. In this work, we analyze how our proposed general-purpose learning framework-the developmental network-2 (DN-2)-autonomously generates internal hierarchy with new mechanisms. Specifically, DN-2 incrementally allocates neuronal resources to different levels of representation during learning instead of handcrafting static boundaries among different levels of representation. We present the mathematical proof to demonstrate that optimal properties in terms of maximum likelihood (ML) are established under the conditions of limited learning experience and resources. The phoneme recognition and real-world visual navigation experiments that are of different modalities and include many different situations are designed to investigate general-purpose learning capability of DN-2. The experimental results show that DN-2 successfully learns different tasks. The formed internal hierarchical representations focus on important features, and the invariant abstract arise from optimal internal representations. We believe that DN-2 is in the right way toward fully autonomous learning.