Image-Matching Based Identification of Store Signage Using Web-Crawled Information

Abstract

We address automatic matching of street images with relevant web resources to enable the identification of store signage in street images. Identification methods for signage usually involve image matching, which attempts to match query images to other similar viewings using pre-labeled copies from a target data set. Manual target data set, such as a fingerprinting database can ensure high-quality data but collected data must be fed manually, which significantly adds costs. Utilizing web-crawled information is a way for automatic data set generation at lower cost, however, imbalanced and noisy data can adversely affect identification accuracy. Our work aims to resolve these issues. We propose a signage identifier in Web-crawled information – SIWI. The SIWI includes a web image data set construction method, which can self-generate high-quality data sets through automated web-mining, including data filtering and pruning strategies, which effectively reduce the identification error caused by noise, imbalance, and insufficient data. Furthermore, by applying a Hybrid Image Matching method that combines the deep learning approach with the feature point matching to signage identification without Optical Character Recognition, it can handle arbitrary signage designs. Because there is no specialized training involved, the same process should also work for any other locations without manual adjustment. An experimental result achieves 91% accuracy in a real-life application, which confirms its effectiveness.