How to Wear Beautifully? Clothing Pair Recommendation

Abstract

In this paper, we present a practical system to automatically suggest the most pairing clothing items, given the reference clothing (upper-body or low-body). This has been a challenge, due to clothes having a variety of categories. Clothing is one of the most informative cues for human appearance. In our daily life, people need to wear properly and beautifully to show their confidence, politeness and social status in various occasions. However, it is not easy to decide what to wear and how to coordinate their own clothes. To address this problem, we propose a recommendation approach that includes clothing region detection, clothing pair recommendation and distance fusion. Clothing region detection based on Faster R-CNN is used to detect clothing region. Clothing pair recommendation consists of a quadruple network architecture, where one dual network of the architecture adopts Siamese convolution neural network architecture. Training examples are pairs of upper-body and low-body clothing items that are either compatible or incompatible. The other dual network is used to learn clothing style features of the input image. This framework is designed to learn a feature transformation from the images of clothing items into two latent spaces, which we call them compatible space and style space respectively. After training the two dual networks, we use a distance fusion method to fuse the features extracted from the compatible and style dual networks. To acquire an optimized model and verify our proposed method, we expand an existing large clothing dataset WoG (Weather-to-Garment), and the resulted dataset is called “How to Wear Beautifully” (H2WB). Experiments on the H2WB dataset demonstrate that our approach is effective with clothing region detection and clothing pair recommendation as well as distance fusion.