Socio-mobile landmark recognition using local features with adaptive region selection

Abstract

With the fast development of mobile devices as well as the broadband wireless network, mobile devices are playing a more and more important role in people׳s daily life. Nowadays, many landmark images are captured by mobile devices. However, these images are often captured under different lightening conditions with varied poses and camera orientations. Besides, people are inherently connected by personal interests as well as various interactions. To alleviate the imaging problem with mobile devices as well as take advantage of the social information for mobile visual applications, we propose a novel socio-mobile visual recognition method using local features with adaptive region selection. We densely extract local regions and use the pixel gradients to represent each local region. Each local region is divided into 4×4 subregions to combine the spatial information. Instead of using fixed pixel numbers for each subregion, we adaptively choose the proper size of each subregion to cope with varied poses and camera orientations. The most discriminative local features are then chosen by minimizing the sparse coding loss. Besides, a geo-discriminative codebook is also generated to take advantages of images׳ location information. Moreover, we jointly consider the visual distances as well as user׳s friends׳ matching results to further boost the final visual recognition performance. We achieve the state-of-the-art performance on the Stanford mobile visual search dataset and the San Francisco landmark dataset. These experimental results demonstrate the effectiveness and efficiency of the proposed adaptive region selection based local features for socio-mobile landmark recognition.