Identifying Places Using Multimodal Social Network Data

Abstract

AbstractSocial media is an interesting source of information, specially when physical sensors are not available. In this paper, we explore several methodologies for the geolocation of multimodal information (image and text) from social networks. To this end, we use pre-trained neural network models for the classification of images and their associated texts. The result is a system that allows creating new synergies between image and text in order to geolocate information that has not been previously geotagged by any other way, which is a potentially relevant information for several purposes. Different experiments have been done revealing that, in general, text information is more accurate and relevant than images.KeywordsMultimodal classificationLocation-based retrievalTransformersSocial networks