Clothescounter: A framework for star-oriented clothes mining from videos

Abstract

This paper presents a novel framework, ClothesCounter, which aims to automatically identify clothes worn by certain stars in videos. At first, several deep convolutional neural networks (CNN) models were utilized to preprocess video data in order to detect clothing images from original video frames, including human body detection, human posture selection, human pose estimation, face verification, and clothing detection. We then propose a method for extracting features of clothing images based on triplet loss that can map clothing images into a compact feature space. In the learned feature space, we present a two-stage clustering algorithm that does not require the number of clusters. Our framework was examined in a large-scale video dataset. Experimental results demonstrate the feasibility and effectiveness of our proposed method.