Probabilistic generative modeling and reinforcement learning extract the intrinsic features of animal behavior

Abstract

It is one of the ultimate goals of ethology to understand the generative process of animal behavior, and the ability to reproduce and control behavior is an important step in this field. However, it is not easy to achieve this goal in systems with complex and stochastic dynamics such as animal behavior. In this study, we have shown that MDN–RNN,a type of probabilistic deep generative model, is able to reproduce stochastic animal behavior with high accuracy by modeling the behavior of C. elegans. Furthermore, we found that the model learns different dynamics in a disentangled representation as a time-evolving Gaussian mixture. Finally, by combining the model and reinforcement learning, we were able to extract a behavioral policy of goal-directed behavior in silico, and showed that it can be used for regulating the behavior of real animals. This set of methods will be applicable not only to animal behavior but also to broader areas such as neuroscience and robotics.