Parsing fashion image into mid‐level semantic parts based on chain‐conditional random fields

Abstract

In this study, the authors address the problem of parsing fashion images into mid-level semantic parts including upper-clothing, lower-clothing, skin, hair and background. These mid-level parts provide the regional information of fashion items and have potential value in high-level parsing process. The key idea of the method is to parse the mid-level parts by region expanding. Owing to the co-occurrence of pose skeleton and the proposed parts, the region expanding process starts from the super-pixels crossed by specific segments of pose skeleton. The super-pixels are then merged with their neighbours by conditional inference based on their position and perceptual similarity. To avoid the difficulties of training on arbitrary graph structures, conditional random fields (CRFs) are constructed on super-pixel chains, which are extracted from the generated expanding trees. This is followed by a voting stage to mix up the probabilities estimated by the chain-CRFs to obtain the final result. Experiments on two datasets show that the new method outperforms related approaches in regional accuracy and has good generalisation capability. Furthermore, the method can be easily employed to improve the performance of high-level parsing. Its effectiveness has been verified by another group of experiments on two state-of-the-art high-level parsing approaches.