Geolocalization of Crowdsourced Images for 3-D Modeling of City Points of Interest

Abstract

Geolocalization of crowdsourced images is a challenging task that is getting increased attention nowadays due to the rise in popularity of geotagging and its applications. Among these applications, 3-D modeling from Internet photograph collections is a very active research topic with great promise and potential. In order to automize and optimize the crowdsourced 3-D modeling process, this letter proposes a novel framework that can be used for automatic 3-D modeling of city points of interest (POIs), such as statues, buildings, and temporary artworks. Crowdsourced images related to the POI and its location are collected using a geographical Web search process based on geotags and semantic geodata. Subsequently, panoramic Google Street View (SV) images are used to geolocalize the images. If enough feature matches are found between the image and one of the SV images, the image is annotated with the location metadata of the best matching image from the SV database. Otherwise, when too few matches are found, the image most probably will not contain the POI in its field of view (FOV), and it is filtered out. For optimal performance, the equirectangular panoramic SV images are transformed into an SV data set of perspective cutouts facing the POI with different pitches and FOVs. From this data set, a basic 3-D model of the POI and its environment is generated. Finally, the geolocalized crowdsourced images refine and optimize the 3-D model using the matching matrix that is generated from the geolocalization results. Experiments show the feasibility of our approach on different types of city POIs. Our main contribution is that we can decrease the 3-D modeling computation time by more than half and significantly improve the model completeness. Finally, it is important to remark that the applicability of the proposed framework is not limited to 3-D modeling but can also be used in other domains, such as geoaugmented reality and location-based media annotation.