Big Data-Driven Pedestrian Analytics: Unsupervised Clustering and Relational Query Based on Tencent Street View Photographs

Fan Xue,Xiao Li,Zhe Chen,Weisheng Lu,Lvwen Lin,Christopher J Webster

doi:10.3390/ijgi10080561

Abstract

Recent technological advancements in geomatics and mobile sensing have led to various urban big data, such as Tencent street view (TSV) photographs; yet, the urban objects in the big dataset have hitherto been inadequately exploited. This paper aims to propose a pedestrian analytics approach named vectors of uncountable and countable objects for clustering and analysis (VUCCA) for processing 530,000 TSV photographs of Hong Kong Island. First, VUCCA transductively adopts two pre-trained deep models to TSV photographs for extracting pedestrians and surrounding pixels into generalizable semantic vectors of features, including uncountable objects such as vegetation, sky, paved pedestrian path, and guardrail and countable objects such as cars, trucks, pedestrians, city animals, and traffic lights. Then, the extracted pedestrians are semantically clustered using the vectors, e.g., for understanding where they usually stand. Third, pedestrians are semantically indexed using relations and activities (e.g., walking behind a guardrail, road-crossing, carrying a backpack, or walking a pet) for queries of unstructured photographic instances or natural language clauses. The experiment results showed that the pedestrians detected in the TSV photographs were successfully clustered into meaningful groups and indexed by the semantic vectors. The presented VUCCA can enrich eye-level urban features into computational semantic vectors for pedestrians to enable smart city research in urban geography, urban planning, real estate, transportation, conservation, and other disciplines.

Highlights

A city’s information infrastructure, which measures and extracts valuable data from the multi-faceted urban systems, is the foundation for enabling smart solutions for urban dwellers and defining public administration efficiency [1,2]
This study investigates big data-driven semantic vectors for processing a street view database; we use the DeepLab V3 model to conduct pixel-level semantic segmentation [52], which performs better than pyramid scene parsing network (PSPNet) with an 81.3% mean IoU
All the 530,000 Tencent street view (TSV) photographs were processed by the convolutional neural network (CNN) and R-CNN deep learning models in 22 days

Summary

Introduction

A city’s information infrastructure, which measures and extracts valuable data from the multi-faceted urban systems, is the foundation for enabling smart solutions for urban dwellers and defining public administration efficiency [1,2]. Only a certain degree of integration and understanding of big data and turning it into knowledge and smartness can lead to the realization of more sustainable urban environments in smart cities [7]. Big data at a city-scale can help people understand the dynamic status of urban stock and flow objects, systems, and operations and assist in making agile stock, flow and overall systems management decisions, thereby improving resources allocation, cutting urban operation costs, and fostering a more sustainable living environment [24]. Urban big data analytics for smart cities can benefit various domains, including transportation and logistics, energy consumption and resources, construction and buildings, public governance and environment, healthcare and education, social welfare and the economy [3]

Objectives

Methods

Results

Discussion

Conclusion