Muscle Vectors as Temporally Dense "Labels"

Abstract

Consider a human who interacts with the physical world to autonomously learn in a task non-specific way through lifetime. It seems obvious that the fully autonomous learner does not have the luxury to have the mother to provide temporally dense state labels, but the context/state at every frame is beneficial (e.g., to generate attention for the next frame). How can we enable the learner to generate frame-wise contexts/states on the fly? Our past work on Developmental Network (DN-1) has shown that frame-wise state labels (e.g., stages within a phoneme) are useful to generate temporally sparse label (the type of the phoneme). However, such dense and sparse labels were handcrafted from a static data set, using human identified frame-wise equivalence. In this paper, we study a conceptually challenging problem — how to enable an autonomous learner to generate frame-wise states autonomously without human handcrafting dense labels at all. We propose that frame-wise muscle actions (e.g., producing a sound) are not only temporally dense and high-dimensional, but also natural as dense labels. However, it is unknown how a neural network can use such high-dimensional vectors as dense labels. In this work, we provide a model for this new issue and experiment with Developmental Network-2 (DN-2) for imitation of audio sequences. Our experimental results showed DN-2 can successfully emerge high-dimensional real-valued vector actions. These actions provide DN-2 with frame-wise temporal context information. This work corresponds to a key step toward our goal to enable the agent to fully autonomously generate frame-wise actions without human-provided dense labels and with only a few human-provided sparse labels.