Predicting environmental features by learning spatiotemporal embeddings from social media

Abstract

Spatiotemporal modelling is an important task for ecology. Social media tags have been found to have great potential to assist in predicting aspects of the natural environment, particularly through the use of machine learning methods. Here we propose a novel spatiotemporal embeddings model, called SPATE, which is able to integrate textual information from the photo-sharing platform Flickr and structured scientific information from more traditional environmental data sources. The proposed model can be used for modelling and predicting a wide variety of ecological features such as species distribution, as well as related phenomena such as climate features. We first propose a new method based on spatiotemporal kernel density estimation to handle the sparsity of Flickr tag distributions over space and time. Then, we efficiently integrate the spatially and temporally smoothed Flickr tags with the structured scientific data into low-dimensional vector space representations. We experimentally show that our model is able to substantially outperform baselines that rely only on Flickr or only on traditional sources.