Crowdsourced Photos Research Articles

Mapping of user-perceived landscape types and spatial distribution using crowdsourced photo data and machine learning: Focusing on Taeanhaean National Park

User perception of protected areas is a valuable set of information for monitoring and managing those areas. To refer to the management direction of protected areas, various studies have been conducted to identify users' perspectives and find their needs. This study aimed to propose a method to analyze the landscapes perceived by users in Taeanhaean National Park and their spatial distribution by applying crowdsourced photos created and uploaded based on user experiences, Google Vision API, Latent Dirichlet allocation, and the self-organizing map (SOM) machine learning technique. According to the results, Taeanhaean National Park contains 12 major landscape types that include coastal landscapes, forest landscapes, sand dunes, horizontal land landscapes, and urban landscapes. Using crowdsourced photo data, this study was able to identify landscape types that were unrecognized from the perspective of experts or officials and was also able to map the distribution of each landscape type based on the photos' location information. Furthermore, this study aimed to derive the major bases of several landscape types in the large study area through SOM analysis. By using social media data to analyze users’ preferences and perceptions of national parks and large protected areas, this study proposed a potential indicator for a user-participatory landscape monitoring system. Management implicationsThis study aims to help formulate a management plan considering users’ preferences and perceptions of protected areas using crowdsourced photo data. In the analysis stage, an unsupervised learning method, a type of machine learning, was used to derive landscape areas containing the characteristics of various landscape elements. The study has elucidated the perception of new landscape resources and spaces that the existing analysis could not reveal. Mapping the landscape areas can contribute to the monitoring system of the protected area.∙Taeanhaean National Park comprises 12 Landscape types by analyzing users' perceptions using crowdsourced photo data.∙Four landscape areas were identified using a self-organizing map.∙Social media data and machine learning can be useful for monitoring users' perceptions of national parks.

Automated construction of Wi-Fi-based indoor logical location predictor using crowd-sourced photos with Wi-Fi signals

Owing to the recent proliferation of smartphones and the SNS, a large number of images taken by smartphones at various places have been uploaded to SNSs. In addition, smartphones are equipped with various sensors such as Wi-Fi modules that enable us to generate an image associated with the sensory information that represents the context in which the image was captured. This study demonstrates the benefits of images associated with Wi-Fi signals in the automated construction of a Wi-Fi-based indoor logical location classifier that predicts a semantic location label of a user’s position for shopping complexes. In this study, a logical location class refers to the store class label in a shopping complex, such as Starbucks and H&M. Given a collection of images associated with Wi-Fi signals taken at a shopping complex and the complex’s floor plan, the proposed method first estimates the store label at which an image was taken by analyzing the image and crawled online images of branch stores. Then, the 2D coordinates of the images taken at branch stores on the floor coordinate system can be estimated using the floor plan. Subsequently, by using the Wi-Fi signals of the branch store images and their estimated 2D coordinates, we construct a transformation function that maps Wi-Fi signals onto the 2D coordinates, and we adopt this function to predict an indoor location class of an observed Wi-Fi scan from a smartphone possessed by an end user. The proposed transformation function comprises an ensemble of sub-functions designed based on CVAEs. Finally, we demonstrate the effectiveness of the proposed method for three actual shopping complexes.

Citizen Science to Assess Light Pollution with Mobile Phones

The analysis of the colour of artificial lights at night has an impact on diverse fields, but current data sources have either limited resolution or scarce availability of images for a specific region. In this work, we propose crowdsourced photos of streetlights as an alternative data source: for this, we designed NightUp Castelldefels, a pilot for a citizen science experiment aimed at collecting data about the colour of streetlights. In particular, we extract the colour from the collected images and compare it to an official database, showing that it is possible to classify streetlights according to their colour from photos taken by untrained citizens with their own smartphones. We also compare our findings to the results obtained from one of the current sources for this kind of study. The comparison highlights how the two approaches give complementary information about artificial lights at night in the area. This work opens a new avenue in the study of the colour of artificial lights at night with the possibility of accurate, massive and cheap data collection.

Crowd-sourced SfM: Best practices for high resolution monitoring of coastal cliffs and bluffs

Structure from motion (SfM) photogrammetry is an increasingly common technique for measuring landscape change over time by deriving 3D point clouds and surface models from overlapping photographs. Traditional change detection approaches require photos that are geotagged with a differential GPS (DGPS) location, which requires expensive equipment that can limit the ability of communities and researchers to perform frequent (i.e. daily, weekly, and/or monthly) surveys. Crowd-sourced photos can lower the barrier to entry and substantially increase the frequency of surveys, although such photos often lack accurate location information and can vary in quality. This paper presents a SfM approach for monitoring environmental change in high relief coastal environments that does not require all photos have DGPS location information and does not require field survey data. A 1.5 km section of coastal bluffs near the Elwha River Delta (Washington state) is used to demonstrate the efficacy of this approach. Photos of the bluff were collected with a digital SLR camera or phone camera while either on foot along the beach or from a boat as part of monitoring following removal of two large dams along the Elwha River during 2011–2013. Only 33% of photos had DGPS location information, whereas most photos had no location information or locations that were accurate to a couple of meters. All photos were processed using 3D, 4D, and fixed-floating (FF) SfM alignment methods and the resulting dense point clouds are used to compare the different alignment approaches with crowd-sourced photo sets. Results demonstrate that 4D and FF approaches are more likely to reconstruct and are more accurate than the 3D approach. While the 4D and FF have comparable accuracies, the FF approach is several orders of magnitude more efficient, as this method can leverage camera location information from relatively few photos to improve the accuracy of all aligned and derived products. Effectively utilizing crowd-sourced photos in SfM change detection can improve the frequency of surveying a landscape in a more cost-effective approach that also has potential for citizen-science engagement and communication. This is especially important for data-poor environments such as high-relief coastal cliffs and bluffs, where near-nadir imagery and LIDAR may fail to accurately capture near-vertical cliffs or bluff faces. Based on the analysis of different photo alignment and filtering approaches, we present suggested best practices for engaging citizen scientists in coastal cliff and bluff monitoring efforts through collecting photos amenable for SfM reconstruction.

Building Facade Reconstruction Using Crowd- Sourced Photos and Two-Dimensional Maps

To address the high-cost problem of the current three-dimensional (<small>3D</small>) reconstruction for urban buildings, a new technical framework is proposed to generate <small>3D</small> building facade information using crowd-sourced photos and two-dimensional (2D) building vector data in this paper. The crowd-sourced photos mainly consisted of Tencent street view images and other-source photos, which were collected from three platforms, including search engines, social media, and mobile phones. The photos were selected and grouped first, and then a structure from motion algorithm was used for <small>3D</small> reconstruction. Finally, the reconstructed point clouds were registered with 2D building vector data. The test implementation was conducted in the Jianye District of Nanjing, China, and the generated point clouds showed a good fit with the true values. The proposed <small>3D</small> reconstruction method represents a multi-sourced data integration process. The advantage of the proposed approach lies in the open source and low-cost data used in this study.

POTENTIAL OF GEOLOCATED CROWDSOURCED IMAGE POSTS IN PREDICTING AN EARLY ESTIMATE OF THE PATTERNS OF STRUCTURAL DAMAGE FOLLOWING A HURRICANE

Abstract. During a disaster, the activity of the crowd represents a very valuable source of the on-the-ground conditions shared by the affected citizens. The approach, presented in the paper, explores the relationship between the spatial distribution of crowdsourced image posts and damaged buildings in order to understand the potential of modelling the spatial distribution of damaged buildings based on geolocated images. The posts related to the hurricane Michael that happened in the United States in October 2018, showing the building damage of Panama City, have been collected by NAPSG Foundation and GISCorps volunteers. The building damage assessment, based on the analysis of high-resolution post-event imagery, has been performed by FEMA. Exploring the two available independent point datasets, the spatial pattern of each individual dataset has been analysed and furthermore the spatial relationship between them has been explored. A set of spatial statistics has been performed with R software. For this purpose, the distance-based methods have been used, that consider the mutual position of points to describe the patterns. The results shown the spatial relationship between the crowdsourced photos and different damage types. Furthermore, potential of crowdsourced images for improving the awareness of the structural damage after the hurricane have been discussed.

The Advent of the 4D Mirror World

The 4D Mirror World is considered to be the next planetary-scale information platform. This commentary gives an overview of the history of the converging trends that have progressively shaped this concept. It retraces how large-scale photographic surveys served to build the first 3D models of buildings, cities, and territories, how these models got shaped into physical and virtual globes, and how eventually the temporal dimension was introduced as an additional way for navigating not only through space but also through time. The underlying assumption of the early large-scale photographic campaign was that image archives had deeper depths of latent knowledge still to be mined. The technology that currently permits the advent of the 4D World through new articulations of dense photographic material combining aerial imagery, historic photo archives, huge video libraries, and crowd-sourced photo documentation precisely exploits this latent potential. Through the automatic recognition of “homologous points,” the photographic material gets connected in time and space, enabling the geometrical computation of hypothetical reconstructions accounting for a perpetually evolving reality. The 4D world emerges as a series of sparse spatiotemporal zones that are progressively connected, forming a denser fabric of representations. On this 4D skeleton, information of cadastral maps, BIM data, or any other specific layers of a geographical information system can be easily articulated. Most of our future planning activities will use it as a way not only to have smooth access to the past but also to plan collectively shared scenarios for the future.

Inferring the level of visibility from hazy images

In our research, we would like to exploit crowdsourced photos from social media to create low-cost fire disaster sensors. The main problem is to analyse how hazy the environment looks like. Therefore, we provide a brief survey of methods dealing with visibility level of hazy images. The methods are divided into two categories: single-image approach and learning-based approach. The survey begins with discussing single image approach. This approach is represented by visibility metric based on contrast-to-noise ratio (CNR) and similarity index between hazy image and its dehazing image. This is followed by a survey of learning-based approach using two contrast approaches that is: 1) based on theoretical foundation of transmission light, combining with the depth image using new deep learning method; 2) based on black-box method by employing convolutional neural networks (CNN) on hazy images.

Spotlight: Hot target discovery and localization with crowdsourced photos

Camera-equipped mobile devices are encouraging people to take more photos and the development and growth of social networks is making it increasingly popular to share photos online. When objects appear in overlapping Fields Of View (FOV), this means that they are drawing much attention and thus indicates their popularity. Successfully discovering and locating these objects can be very useful for many applications, such as criminal investigations, event summaries, and crowdsourcing-based Geographical Information Systems (GIS). Existing methods require either prior knowledge of the environment or intentional photographing. In this paper, we propose a seamless approach called “Spotlight”, which performs passive localization using crowdsourced photos. Using a graph-based model, we combine object images across multiple camera views. Within each set of combined object images, a photographing map is built on which object localization is performed using plane geometry. We evaluate the system's localization accuracy using photos taken in various scenarios, with the results showing our approach to be effective for passive object localization and to achieve a high level of accuracy.

Mapping cultural ecosystem services 2.0 – Potential and shortcomings from unlabeled crowd sourced images

The volume of accessible geotagged crowdsourced photos has increased. Such data include spatial, temporal, and thematic information on recreation and outdoor activities, thus can be used to quantify the demand for cultural ecosystem services (CES). So far photo content has been analyzed based on user-labeled tags or the manual labeling of photos. Both approaches are challenged with respect to consistency and cost-efficiency, especially for large-scale studies with an enormous volume of photos. In this study, we aim at developing a new method to analyze the content of large volumes of photos and to derive indicators of socio-cultural usage of landscapes. The method uses machine-learning and network analysis to identify clusters of photo content that can be used as an indicator of cultural services provided by landscapes. The approach was applied in the Mulde river basin in Saxony, Germany. All public Flickr photos (n = 12,635) belonging to the basin were tagged by deep convolutional neural networks through a cloud computing platform, Clarifai. The machine-predicted tags were analyzed by a network analysis that leads to nine hierarchical clusters. Those clusters were used to distinguish between photos related to CES (65%) and not related to CES (35%). Among the nine clusters, two clusters were related to CES: ‘landscape aesthetics’ and ‘existence’. This step allowed mapping of different aspects of CES and separation of non-relevant photos from further analysis. We further analyzed the impact of protected areas on the spatial pattern of CES and not-related CES photos. The presence of protected areas had a significant positive impact on the areas with both ‘landscape aesthetics’ and ‘existence’ photos: the total number of days in each mapping unit where at least one photo was taken by a user (‘photo-user-day’) increased with the share of protected areas around the location. The presented approach has shown its potential for reliable mapping of socio-cultural uses of landscapes. It is expected to scale well with large numbers of photos and to be easily transferable to different regions.

Mapping the intangible: Using geolocated social media data to examine landscape aesthetics

The ecosystem services concept is increasingly gaining momentum in land-use policies and landscape planning. Yet, cultural ecosystem services often lack proper assessments. With this study, we use novel methodological approaches to map the cultural ecosystem service landscape aesthetics for its enhanced consideration in land-use policies. Our study uses expert-based participatory mapping and crowd-sourced (social media) photo data to examine the spatial distribution of landscape aesthetics in the Province of Barcelona, Catalonia. We distinguish the capacity and flow of landscape aesthetics. Landscape aesthetics capacity was assessed through spatial multi-criteria evaluation, consisting of a viewshed analysis and an expert-based selection and weighting of landscape features. Landscape aesthetics flow, i.e., people’s actual appreciation of landscape aesthetics, was assessed by analysing a sample of 13,460 geolocated photographs from the social media platform Flickr. Our results uncover a substantial mismatch between landscape aesthetics capacity and flow. While landscape aesthetics capacity is widely distributed across the case study area, landscape aesthetics flow is (with few exceptions) mostly concentrated in urban and periurban areas. The main insights for land-use policies derived from our results are twofold. On one hand, landscape aesthetics flow seems less dependent on ‘pristine nature’ than experts and planners assume, while the complex integration of green and grey landscape features plays a critical role. On the other hand, urban and periurban landscapes as key landscape aesthetics providers should receive additional attention in land-use policies.

Using crowd-sourced photos to assess seasonal patterns of visitor use in mountain-protected areas.

Managing protected areas effectively requires information about patterns of visitor use, but these data are often limited. We explore how geotagged photos on Flickr, a popular photo-sharing social-media site, can generate hotspot maps and distribution models of temporal and spatial patterns of use in two mountain-protected areas of high conservation value. In Aconcagua Provincial Park (Argentina), two routes to the summit of Aconcagua were used in summer, but most visitors stayed close to the main road, using formal and informal walking trails and the Visitor Centre, while in winter, there was very limited visitation. In Kosciuszko National Park (Australia), alpine walking trails were popular in summer, but in winter, most visitors stayed in the lower altitude ski resorts and ski trails. Results demonstrate the usefulness of social-media data alone as well as a complement for visitor monitoring, providing spatial and temporal information for site-specific and park-level management of visitors and potential impacts in conservation areas.

Inferring the Level of Visibility from Hazy Images

In our research, we would like to exploit crowdsourced photos from social media to create low-cost fire disaster sensors. The main problem is to analyse how hazy the environment looks like. Therefore, we provide a brief survey of methods dealing with visibility level of hazy images. The methods are divided into two categories: single-image approach and learning-based approach. The survey begins with discussing single image approach. This approach is represented by visibility metric based on contrast-to-noise ratio (CNR) and similarity index between hazy image and its dehazing image. This is followed by a survey of learning-based approach using two contrast approaches that is: 1) based on theoretical foundation of transmission light, combining with the depth image using new deep learning method; 2) based on black-box method by employing convolutional neural networks (CNN) on hazy images.

Efficient video collection association using geometry-aware Bag-of-Iconics representations

Recent years have witnessed the dramatic evolution in visual data volume and processing capabilities. For example, technical advances have enabled 3D modeling from large-scale crowdsourced photo collections. Compared to static image datasets, exploration and exploitation of Internet video collections are still largely unsolved. To address this challenge, we first propose to represent video contents using a histogram representation of iconic imagery attained from relevant visual datasets. We then develop a data-driven framework for a fully unsupervised extraction of such representations. Our novel Bag-of-Iconics (BoI) representation efficiently analyzes individual videos within a large-scale video collection. We demonstrate our proposed BoI representation with two novel applications: (1) finding video sequences connecting adjacent landmarks and aligning reconstructed 3D models and (2) retrieving geometrically relevant clips from video collections. Results on crowdsourced datasets illustrate the efficiency and effectiveness of our proposed Bag-of-Iconics representation.

SmartPhoto: A Resource-Aware Crowdsourcing Approach for Image Sensing with Smartphones

Photos obtained via crowdsourcing can be used in many critical applications. Due to the limitations of communication bandwidth, storage, and processing capability, it is a challenge to transfer the huge amount of crowdsourced photos. To address this problem, we propose a framework, called SmartPhoto , to quantify the quality (utility) of crowdsourced photos based on the accessible geographical and geometrical information (called metadata ) including the smartphone's orientation, position, and all related parameters of the built-in camera. From the metadata, we can infer where and how the photo is taken, and then only transmit the most useful photos. Four optimization problems regarding the tradeoffs between photo utility and resource constraints, namely Max-Utility, online Max-Utility, Min-Selection, and Min-Selection with $k$ -coverage, are studied. Efficient algorithms are proposed and their performance bounds are theoretically proved. We have implemented SmartPhoto in a testbed using Android based smartphones, and proposed techniques to improve the accuracy of the collected metadata by reducing sensor reading errors and solving object occlusion issues. Results based on real implementations and extensive simulations demonstrate the effectiveness of the proposed algorithms.

Visualizing the perceived environment using crowdsourced photo geodata

Assessing information on aspects of identification, perception, emotion, and social interaction with respect to the environment is of particular importance to the fields of natural resource management. Our ability to visualize this type of information has rapidly improved with the proliferation of social media sites throughout the Internet in recent years. While many methods to extract information on human behavior from crowdsourced geodata already exist, this paper focuses on visualizing landscape perception for application to the fields of landscape and urban planning. Visualization of peoples’ perceptual responses to landscape is demonstrated with crowdsourced photo geodata from Flickr, a popular photo sharing community. A basic, general method to map, visualize, and evaluate perception and perceptual values is proposed. The approach utilizes common tools for spatial knowledge discovery and builds on existing research, but is specifically designed for implementation within the context of landscape perception analysis and particularly suited as a base for further evaluation in multiple scenarios. To demonstrate the process in application, three novel types of visualizations are presented: the mapping of sightlines in Yosemite Valley, the assessment of landscape change in the area surrounding the High Line in Manhattan, and individual location analysis for Coit Tower in San Francisco. The results suggest that analyzing crowdsourced data may contribute to a more balanced assessment of the perceived landscape, which provides a basis for a better integration of public values into planning processes.