Attribute-aware heterogeneous graph network for fashion compatibility prediction

Abstract

Listen

Translate article

Fashion compatibility prediction aims to provide a compatibility score for a set of fashion combinations, making an effort to meet people’s needs for clothing matching in daily life. One difficulty of this problem is that whether the fashion items are compatible depends on a variety of attribute factors, such as design style, material, color. Existing works mostly use the attributes as the item feature representations or integrate attribute features into the compatibility modeling process as auxiliary information in a simple way. These works ignore the fine-grained attributes that can play a role as bridges in establishing the potential compatibility relationship between fashion items. To better take advantage of fashion attributes information, we propose an Attribute-aware Heterogeneous Graph Network (AHGN), which combines the fashion items and their attributes as a heterogeneous graph to integrate the heterogeneous information in the fashion compatibility problem. While fusing the fine-grained attribute information into the item nodes representation, the attributes act as bridges to transfer the characteristics of potential compatible fashion items into the target item node. Previous methods of calculating outfit score by item pairs without considering the effect of outfit compositions. An outfit-based item pairs scoring method is proposed. The feature representation of item pairs is adjusted by integrating the overall outfit information to consider the influence of the outfit environment on the item pairs. We conduct experiments on three tasks (fill-in-the-blank, compatibility prediction, and fashion item retrieval) in two real-world fashion datasets: Fashion32 and IQON3000. Experimental results demonstrate the effectiveness of our proposed method.