Neural stylist: Towards online styling service

Abstract

Online stylist service enjoys huge economic potentials due to the trend of transformation of the fashion industry to digitalisation. Existing works either predict the fashion compatibility from the overall aspect or evaluate the compatibility with type-conditional representations. The prediction is hard to interpret due to the abstractive forecast. This paper proposes a visual and semantic representation model for explainable evaluation and recommendation. The model considers fashion compatibility from different factors, such as colour, material and style, by leveraging low to high-level features from former to later layers of CNN. The colour correlation and the pairwise relationship of fashion items in the same outfit are considered during the prediction stage. Instead of just predicting an outfit as compatible or incompatible, the model can classify an outfit as three precise evaluation levels: Good, Normal and Bad. The detailed compatible level is more consistent with the fashion sense of our human brain as Good or Bad outfits may have specific characteristics while Normal outfits tend to be ordinary. Additionally, the model can diagnose and recommend substitutions of the problematic fashion items from overall compatibility or colour-specific aspects by tracking the prediction matrices’ backpropagation gradients during the recommendation stage. Experiments in terms of outfit compatibility prediction and fill in the blank are conducted to evaluate the prediction ability of the proposed model. In contrast, fashion substitution recommendation experiments are conducted to assess the compatibility diagnosis and recommendation ability. Quantitative and qualitative results show that the model enables online stylist services with excellent explainability and generalisation on fashion prediction and recommendation.