Development of Convolutional Neural Networks for Analyzing Game Icon and Screenshot Images

Abstract

Icons and screenshots are important media displayed in game distribution platforms for providing a brief understanding of the game content to the customers. In this study, we develop ensemble convolutional neural networks for icon and screenshot analysis as three applications: an automatic genre classification, a similar game searching, and a recognition quality assessment. First, the genre classifier is developed using 154358 images from 18 030 games in 17 genres. The proposed genre classifiers achieve 40.5% and 47.6% accuracies for classifying a single icon and a single screenshot, which outperform the average performance of the human testers. The accuracy can be boosted to 54.2% by aggregating results from every image of the game. The Grad-CAM is applied to analyze what models learned. Then, the feature extraction part trained by this task is transferred to the other two applications. For the similar game searching, a dissimilarity of two images is directly computed by the Euclidean distance in the feature space. We define a dissimilarity between two games which are sets of multiple images based on their image-pairwise dissimilarity. The results show that the features are successfully transferred, and the model seems to be able to cluster the games with a similar gameplay and differentiate them from the other gameplays even if they come from the same genre. For the third application, we develop a system for quality assessment of game images based on the correctness of viewers’ understanding of game content by combining multiple models from three different problem definitions. Our system can identify good-genre-representing game images which most of the human testers can recognize their genre correctly with 75.0% accuracy for icons and 76.2% accuracy for screenshots.