UbiShop: Commercial item recommendation using visual part-based object representation

Abstract

With the popularity of online shopping, people have used to shop commercial items on the online shopping websites for convenience. However, based on traditional text search methods, people usually can not find the interesting commercial item they want if they do not know its detailed information, e.g., the name and the seller. Therefore, a more convenient method to help people find the commercial item they want is desired. In this work, we develop a practical system, UbiShop, on mobile phones, whereby users can timely get the related information of interesting commercial items by taking pictures of them. Users can also obtain recommendations on visually similar commercial items to help their buying selections. With the observation that people's preferences on commercial items usually simply depend on their partial visual styles, we propose a novel representation, Visual Part-based Object Representation (VPOR), for commercial item images. The concept of VPOR is to decompose an item image into a set of disjointed partitions, with each of them represents a meaningful semantic parts. User can thus assign non-uniform preferences on the different parts of the commercial item to obtain a personalized recommended results. The experimental results verify our observation and show that the proposed VPOR based commercial item recommendation can achieve better performance than existing text-based and visual-based methods according to the user study.