Outfit Compatibility Research Articles

Outfit compatibility model using fully connected self-adjusting graph neural network

Outfit compatibility model using fully connected self-adjusting graph neural network

Hierarchical Multimodal Graph Learning for Outfit Compatibility Modelling

Hierarchical Multimodal Graph Learning for Outfit Compatibility Modelling

A Review of Modern Fashion Recommender Systems

The textile and apparel industries have grown tremendously over the past few years. Customers no longer have to visit many stores, stand in long queues, or try on garments in dressing rooms, as millions of products are now available in online catalogs. However, given the plethora of options available, an effective recommendation system is necessary to properly sort, order, and communicate relevant product material or information to users. Effective fashion recommender systems (RSs) can have a noticeable impact on billions of customers’ shopping experiences and increase sales and revenues on the provider side. The goal of this survey is to provide a review of RSs that operate in the specific vertical domain of garment and fashion products. We have identified the most pressing challenges in fashion RS research and created a taxonomy that categorizes the literature according to the objective they are trying to accomplish (e.g., item or outfit recommendation, size recommendation, and explainability, among others) and type of side information (users, items, context). We have also identified the most important evaluation goals and perspectives (outfit generation, outfit recommendation, pairing recommendation, and fill-in-the-blank outfit compatibility prediction) and the most commonly used datasets and evaluation metrics.

Personalized fashion outfit generation with user coordination preference learning

This paper focuses on personalized outfit generation, aiming to generate compatible fashion outfits catering to given users. Personalized recommendation by generating outfits of compatible items is an emerging task in the recommendation community with great commercial value but less explored. The task requires to explore both user-outfit personalization and outfit compatibility, any of which is challenging due to the huge learning space resulted from large number of items, users, and possible outfit options. To specify the user preference on outfits and regulate the outfit compatibility modeling, we propose to incorporate coordination knowledge in fashion. Inspired by the fact that users might have coordination preference in terms of category combination, we first define category combinations as templates and propose to model user-template relationship to capture users’ coordination preferences. Moreover, since a small number of templates can cover the majority of fashion outfits, leveraging templates is also promising to guide the outfit generation process. In this paper, we propose Template-guided Outfit Generation (TOG) framework, which unifies the learning of user-template interaction, user–item interaction and outfit compatibility modeling. The personal preference modeling and outfit generation are organically blended together in our problem formulation, and therefore can be achieved simultaneously. Furthermore, we propose new evaluation protocols to evaluate different models from both the personalization and compatibility perspectives. Extensive experiments on two public datasets have demonstrated that the proposed TOG can achieve preferable performance in both evaluation perspectives, namely outperforming the most competitive baseline BGN by 7.8% and 10.3% in terms of personalization precision on iFashion and Polyvore datasets, respectively, and improving the compatibility of the generated outfits by over 2%.

An efficient framework for outfit compatibility prediction towards occasion

Nowadays, in a communicating society, fashion is an integral part of a human life, and it is more comfortable and confident when people dress well. Outfit compatibility is not only a combination of different items but also regarding various aspects, such as style, user preferences, and specific occasions. Most of the existing works lead to address the outfit compatibility concerning only style or user preferences, and have no regard for occasions. In this paper, we propose an efficient method for both outfit compatibility and the fill-in-the-blank tasks according to specific occasions. To this end, we utilized an auxiliary classification branch to learn the significantly important features regarding specific occasions. Besides, a sequence to sequence approach is also applied to learn the relationship of different items along with a visual semantic space, which is able to learn the connection between visual features and their semantic presentation. To demonstrate the effectiveness of the proposed method, we conduct experiments on our newly collected Shoplook-Occasion dataset. The experimental results indicate that our proposed method improved the AUC metric from 0.02 to 0.15% and from 0.5 to 4% on accuracy, compared with other approaches for outfit compatibility problem conditioning on specific occasions.

Towards private stylists via personalized compatibility learning

Personalized outfit compatibility learning is an emerging yet challenging task. Most of the existing methods focus on general outfit compatibility learning. Although a few works have been proposed for personalized fashion compatibility, they either considered user preference on fashion items with specific patterns or design elements or recommended outfits based on the overall visual similarity according to the users’ preferred collections. This paper adopts physical and fashion attributes for effective personalized fashion compatibility evaluation and recommendation. The physical attributes are concluded into seven aspects: body shape, skin color, hairstyle, hair color, height, breast size (breasts), and color contrast. The personalized outfit compatibility problem in this paper is a multi-label classification problem and formulated as an optimization function with outfit images, fashion attributes, and physical attributes as input. It is the first attempt to solve the problem by discovering the correlation between visual image features, fashion attributes, and physical attributes. Specifically, the correlation is learned with two transformer encoders by updating attention weights of different embedding pairs during the training process. The model can not only predict the fashion attributes of the outfit’s top, bottom, shoes, and bag items, but also predict the incompatible physical attributes of an individual towards the given outfit. It can be used to recommend outfits that best fit an individual and the predicted fashion attributes can be used for result explanation. The O4U dataset, which contains rich annotations of fashion item attributes and human physical attributes of the outfits, is used to evaluate the performance of the proposed method. The quantitative and qualitative results show that the proposed method outperforms state-of-the-art methods for personalized outfit compatibility evaluation.

Modality-Oriented Graph Learning Toward Outfit Compatibility Modeling

Modality-Oriented Graph Learning Toward Outfit Compatibility Modeling

Learning Fashion Compatibility With Context Conditioning Embedding

Fashion compatibility predictions have obtained a lot of attention recently. Mining the compatibility between fashion items in an outfit is different from learning the visual similarity, since this relationship is more delicate. Decomposing the outfit compatibility into pairwise item matching is a popular way to treat the problem. However, in most existing methods, the items are matched without considering the context, i.e, the remaining items in the outfit. Recent efforts have been made to learn the underlying high order relationships among items by treating the outfit as a whole. These models could be sensitive to the properties of different datasets, and the item representations in these models are not as compact as those in the pairwise models. In this paper, we propose a context conditioning embedding approach to learn compact representations that preserve the shared information among items under the existence of contextual items. We use two different spaces, the general and the contextual spaces, to embed items, where the representation in the contextual space contains information from the context. We employ mutual information maximization for model learning, which is shown to be more appropriate for the problem. With extensive experiments, we show that our model achieves superior performance than other state-of-the-art methods.

Diagnosing fashion outfit compatibility with deep learning techniques

Fashion image understanding is a popular research field with many different machine learning applications. There have been many studies regarding outfit prediction and outfit composition in the field of fashion. However, there are few works that explain the prediction. This paper addresses a method of diagnosing outfit compatibility through clothing images. The proposed system not only predicts compatibility, but also diagnoses incompatible clothing items in outfits. First, a new dataset named ModAI, which has clothing images and compatibility comments from different users was created. After this, a common compatibility comment was created according to user comments for each clothing image. Lastly, image captioning techniques were used to generate compatibility suggestion texts from clothing images. Different segmentation techniques were also used to improve captioning capabilities. The model achieves a 0.62 BLEU-4 score. Experiments show that image captioning techniques can also be used to diagnose outfit compatibility.

OCPHN: Outfit Compatibility Prediction with Hypergraph Networks

With the rapid development of the online shopping, the pursuit of outfit compatibility has become a basic requirement for an increasing number of customers. The existing work on outfit compatibility prediction largely focuses on modeling pairwise item compatibility without considering modeling the whole outfit directly. To address the problem, in this paper, we propose a novel hypergraph-based compatibility modeling scheme named OCPHN, which is able to better model complex relationships among outfits. In OCPHN, we represent the outfit as a hypergraph, where each hypernode represents a category and each hyperedge represents the interactions between multiple categories (i.e., they appear in the same outfit). To better predict outfit compatibility, the hypergraph is transformed into a simple graph, and the message propagation mechanism in the graph convolution network is used to aggregate the neighbours’ information on the node and update the node representations. Furthermore, with learned node representations, an attention mechanism is introduced to compute the outfit compatibility score. Using a benchmark dataset, the experimental results show that the proposed method is an improvement over the strangest baselines in terms of accuracy by about 3% and 1% in the fill-in-the-blank and compatibility prediction tasks, respectively.

Neural stylist: Towards online styling service

Online stylist service enjoys huge economic potentials due to the trend of transformation of the fashion industry to digitalisation. Existing works either predict the fashion compatibility from the overall aspect or evaluate the compatibility with type-conditional representations. The prediction is hard to interpret due to the abstractive forecast. This paper proposes a visual and semantic representation model for explainable evaluation and recommendation. The model considers fashion compatibility from different factors, such as colour, material and style, by leveraging low to high-level features from former to later layers of CNN. The colour correlation and the pairwise relationship of fashion items in the same outfit are considered during the prediction stage. Instead of just predicting an outfit as compatible or incompatible, the model can classify an outfit as three precise evaluation levels: Good, Normal and Bad. The detailed compatible level is more consistent with the fashion sense of our human brain as Good or Bad outfits may have specific characteristics while Normal outfits tend to be ordinary. Additionally, the model can diagnose and recommend substitutions of the problematic fashion items from overall compatibility or colour-specific aspects by tracking the prediction matrices’ backpropagation gradients during the recommendation stage. Experiments in terms of outfit compatibility prediction and fill in the blank are conducted to evaluate the prediction ability of the proposed model. In contrast, fashion substitution recommendation experiments are conducted to assess the compatibility diagnosis and recommendation ability. Quantitative and qualitative results show that the model enables online stylist services with excellent explainability and generalisation on fashion prediction and recommendation.

Partially Supervised Compatibility Modeling.

Fashion Compatibility Modeling (FCM), which aims to automatically evaluate whether a given set of fashion items makes a compatible outfit, has attracted increasing research attention. Recent studies have demonstrated the benefits of conducting the item representation disentanglement towards FCM. Although these efforts have achieved prominent progress, they still perform unsatisfactorily, as they mainly investigate the visual content of fashion items, while overlooking the semantic attributes of items (e.g., color and pattern), which could largely boost the model performance and interpretability. To address this issue, we propose to comprehensively explore the visual content and attributes of fashion items towards FCM. This problem is non-trivial considering the following challenges: a) how to utilize the irregular attribute labels of items to partially supervise the attribute-level representation learning of fashion items; b) how to ensure the intact disentanglement of attribute-level representations; and c) how to effectively sew the multiple granulairites (i.e, coarse-grained item-level and fine-grained attribute-level) information to enable performance improvement and interpretability. To address these challenges, in this work, we present a partially supervised outfit compatibility modeling scheme (PS-OCM). In particular, we first devise a partially supervised attribute-level embedding learning component to disentangle the fine-grained attribute embeddings from the entire visual feature of each item. We then introduce a disentangled completeness regularizer to prevent the information loss during disentanglement. Thereafter, we design a hierarchical graph convolutional network, which seamlessly integrates the attribute- and item-level compatibility modeling, and enables the explainable compatibility reasoning. Extensive experiments on the real-world dataset demonstrate that our PS-OCM significantly outperforms the state-of-the-art baselines. We have released our source codes and well-trained models to benefit other researchers (https://site2750.wixsite.com/ps-ocm).

Outfit Compatibility Prediction with an Attribute Detection Model

Outfit Compatibility Prediction with an Attribute Detection Model

Learning compatibility knowledge for outfit recommendation with complementary clothing matching

With the rapid development of mobile networks and e-commerce, clothing recommendation has achieved considerable success in recent years. Fashion outfit matching has become an essential component to users while shopping, which helps users to select and present items to individuals in a personalized fashion recommendation. Apparently, it is an arduous task to guide complementary clothing matching due to the complexity and subjectivity of fashion items. Some existing solutions have been presented in recent years, which are tending to discover a series of visual cues to establish the matching relations. However, it would be mismatched easily due to these methods being hard to represent all the potential semantic information from the appearance of clothes. To thoroughly make use of the visual characteristics of clothing products and the related description information, we propose a complementary clothing matching method with some compatibility knowledge, named it CCMCK shortly. For visual compatibility, we adopt the graph neural network to model the visual relationship between items. To generate an outfit that satisfies the requirement of fashion compatibility, we propose a matching way under the compatibility constraint and seek to recommend compatible items based on multi-modal compatibility. Finally, we performed a qualitative investigation on the fill-in-the-blank and fashion outfit compatibility tasks to evaluate the proposed method.

Attribute-aware explainable complementary clothing recommendation

Modelling mix-and-match relationships among fashion items has become increasingly demanding yet challenging for modern E-commerce recommender systems. When performing clothes matching, most existing approaches leverage the latent visual features extracted from fashion item images for compatibility modelling, which lacks explainability of generated matching results and can hardly convince users of the recommendations. Though recent methods start to incorporate pre-defined attribute information (e.g., colour, style, length, etc.) for learning item representations and improving the model interpretability, their utilisation of attribute information is still mainly reserved for enhancing the learned item representations and generating explanations via post-processing. As a result, this creates a severe bottleneck when we are trying to advance the recommendation accuracy and generating fine-grained explanations since the explicit attributes have only loose connections to the actual recommendation process. This work aims to tackle the explainability challenge in fashion recommendation tasks by proposing a novel Attribute-aware Fashion Recommender (AFRec). Specifically, AFRec recommender assesses the outfit compatibility by explicitly leveraging the extracted attribute-level representations from each item’s visual feature. The attributes serve as the bridge between two fashion items, where we quantify the affinity of a pair of items through the learned compatibility between their attributes. Extensive experiments have demonstrated that, by making full use of the explicit attributes in the recommendation process, AFRec is able to achieve state-of-the-art recommendation accuracy and generate intuitive explanations at the same time.

Outfit compatibility prediction with multi-layered feature fusion network

Clothing plays a critical role in people’s daily lives, a perfect styling clothing is able to help people to avoid weaknesses and show their personal temperament, however, not everyone is good at styling. Compatibility is the core of styling, however, determining whether a pair of garments are compatible with each other is a challenging styling issue. Years of research have been devoted to fashion compatibility learning, whereas there are still several drawbacks in visual feature detection and compatibility calculation. In this paper, we propose an end-to-end framework to learn the compatibility among tops and bottoms. In order to improve the effects of visual feature extraction, a Multi-layer Non-local Feature Fusion framework (MNLFF) is developed. Feature fusion model is used to combine both high and low-level features, while non-local block is for global feature detection. We compare our technique with the prior state-of-the-art methods in the outfit compatibility prediction task and extensive experiments on existing datasets demonstrate its effectiveness.