Abstract
The convolutional neural network (CNN) is widely used in various computer vision problems such as image recognition and image classification because of its powerful ability to process image data. However, it is an end-to-end model that remains a “block box” for users. The internal logic of CNN is not explicitly known. Interpreting CNN can help us better understand neural networks and the various ways they benefit us as users. In this paper, we explain the contributions of the convolutional layer of CNN with a neuroscience experiment paradigm: the Ms. Pac-Man video game. Ms. Pac-Man is a popular game that provides a complex yet natural decision-making task rather than a laboratory artifact. An analysis of the game can thus intuitively reveal the complicated decision-making process in animal brains. We sought to (1) elucidate the role of the CNN convolutional layer and (2) analyze the low-level strategies in animal brains based on high-level decisions. We use recorded videos of monkeys playing the Ms. Pac-Man game to empirically demonstrate that our network is able to predict the moving direction of the Pac-Man at every time step. We further find that the decision-making process at work during gameplay is high-reward-driven. A heatmap of the weighted feature map at each convolutional layer shows that CNN makes predictions based on the most important input pattern, which in this case is the high reward entities in the game.
Highlights
Recent decades have seen remarkable advancements in deep neural network (DNN) technologies in dealing with large-scale and complicated machine learning problems
WORKS In this study, we explored the internal logic of a convolutional neural network (CNN)-based deep neural network
We deployed the network in an analysis of decision-making policies within the neuroscience experiment paradigms of the Ms Pac-Man video game
Summary
Recent decades have seen remarkable advancements in deep neural network (DNN) technologies in dealing with large-scale and complicated machine learning problems. [16] proposed a decision tree structure to quantitatively explain how CNN uses extracted features to classify and how much each filter of the convolutional layer contributes to the final estimation. We observe a CNN at work in a typical neuroscience experiment paradigm, the Ms Pac-Man video game, to explore how animal brains make decisions among multiple alternative choices. In the neuroscience field [30], [31], research has shown that the external behavior of animals is controlled by their brains as they make decisions based on internal representations of new observations This inspired us to integrate CNN with Ms Pac-Man in this study. Because of the end-to-end characteristic of CNN, our network can directly estimate the Pac-Man motion given game images instead of pre-processed statistical or neural data. CNN-BASED VIDEO ANALYSIS FOR PAC-MAN GAME In this study, we used a CNN-based network to analyze the brains’ of animals in making high-level decisions based on recorded videos of them playing the Ms Pac-Man game
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
