Adversarial Attacks Against Visually Aware Fashion Outfit Recommender Systems

Abstract

AbstractPre-trained CNN models are frequently employed for a variety of machine learning tasks, including visual recognition and recommendation. We are interested in examining the application of attacks generated by adversarial machine learning techniques to the vertical domain of fashion and retail products. Specifically, the present work focuses on the robustness of cutting-edge CNN models against state-of-the-art adversarial machine learning attacks that have shown promising performance in general visual classification tasks. In order to achieve this objective, we conducted adversarial experiments on two prominent fashion-related tasks: visual clothing classification and outfit recommendation. Large-scale experimental validation of the fashion category classification task on a real dataset of PolyVore consisting of various outfits reveals that ResNet50 is one of the most resilient networks for the fashion categorization task, whereas DenseNet169 and MobileNetV2 are the most vulnerable. Performance-wise however, DenseNet169 is the most time-consuming network to attack. However, the results of the outfit recommendation task were somewhat unexpected. In both of the push or nuke attack scenarios and altogether, it was demonstrated that adversarial attacks were unable to degrade the quality of outfit recommenders. The only exception was the more complicated adversarial attack of DeepFool, which could only weaken the quality of visual recommenders at large attack budget (\(\epsilon \)) values. Numerous explanations could be provided for this phenomenon, which can be attributed to the fact that a collection of adversarially perturbed images can nonetheless appear pleasing to the human eye. This may possibly be a result of the greater image sizes in the selected dataset. Overall, the results of this study are intriguing and encourage more studies in the field of adversarial attacks and fashion recommendation system security.KeywordsAdversarialAttackFashionRecommender systems