SVI2POI: an end-to-end framework for POI generation from street-view imagery

Recently, many new studies emerged to apply computer vision (CV) to street view imagery (SVI) dataset to objectively extract the view indices of various streetscape features such as trees to proxy urban scene qualities. However, human perceptions (e.g., imageability) have a subtle relationship to visual elements which cannot be fully captured using view indices. Conversely, subjective measures using survey and interview data explain more human behaviors. However, the effectiveness of integrating subjective measures with SVI dataset has been less discussed. To address this, we integrated crowdsourcing, CV, and machine learning (ML) to subjectively measure four important perceptions suggested by classical urban design theory. We first collected experts’ rating on sample SVIs regarding the four qualities which became the training labels. CV segmentation was applied to SVI samples extracting streetscape view indices as the explanatory variables. We then trained ML models and achieved high accuracy in predicting the scores. We found a strong correlation between predicted complexity score and the density of urban amenities and services Point of Interests (POI), which validates the effectiveness of subjective measures. In addition, to test the generalizability of the proposed framework as well as to inform urban renewal strategies, we compared the measured qualities in Pudong to other five renowned urban cores worldwide. Rather than predicting perceptual scores directly from generic image features using convolution neural network, our approach follows what urban design theory suggested and confirms various streetscape features affecting multi-dimensional human perceptions. Therefore, its result provides more interpretable and actionable implications for policymakers and city planners.

Recently, many new studies applying computer vision (CV) to street view imagery (SVI) datasets to objectively extract the view indices of various streetscape features such as trees to proxy urban scene qualities have emerged. However, human perception (e.g., imageability) have a subtle relationship to visual elements that cannot be fully captured using view indices. Conversely, subjective measures using survey and interview data explain human behaviors more. However, the effectiveness of integrating subjective measures with SVI datasets has been less discussed. To address this, we integrated crowdsourcing, CV, and machine learning (ML) to subjectively measure four important perceptions suggested by classical urban design theory. We first collected ratings from experts on sample SVIs regarding these four qualities, which became the training labels. CV segmentation was applied to SVI samples extracting streetscape view indices as the explanatory variables. We then trained ML models and achieved high accuracy in predicting scores. We found a strong correlation between the predicted complexity score and the density of urban amenities and services points of interest (POI), which validates the effectiveness of subjective measures. In addition, to test the generalizability of the proposed framework as well as to inform urban renewal strategies, we compared the measured qualities in Pudong to other five urban cores that are renowned worldwide. Rather than predicting perceptual scores directly from generic image features using a convolution neural network, our approach follows what urban design theory has suggested and confirmed as various streetscape features affecting multi-dimensional human perceptions. Therefore, the results provide more interpretable and actionable implications for policymakers and city planners.

Understanding how street spatial patterns are related to street vitality is conducive to enhancing effective urban and street design. Such analysis is facilitated by big data technology as it enables more accurate methods. This study cites data from street view imagery (SVI) and points of interest (POI) to assess street vitality strength after the classification of street spatial and vitality types to explore the relationship between street spatial patterns and street vitality with a further discussion on the layout features of street vitality and its strength in various street spatial patterns. First, street spatial patterns are quantified based on SVI, which are further classified using principal component analysis and cluster analysis; POI data are then introduced to identify street vitality patterns and layout, and the strength of street vitality is evaluated using spatial overlay analysis. Finally, relevance analysis is explored to cast light on the relationship between street vitality layout and street spatial patterns by overlaying street spatial pattern, street vitality types, and street vitality strength in the grid cells. This paper takes the urban area of Guiyang, China, as an example and the analysis shows that a pattern is discovered in Guiyang regarding the layout of street vitality types and vitality strengths across different street spatial patterns; compact street spaces should be prioritized in designing street space renovation; and cultural leisure vitality is most adaptive to street spatial patterns. Based on big data and using grids to understand the intrinsic relationship between street spatial patterns and the type and strength of street vitality, this paper brings more options to urban street studies in terms of perspective and methodology.

The Manhattan subway system serves 39% of its commuters as an essential public transit option; however, its annual ridership dropped by 3.48% from 2015 to 2018. This study hypothesizes that ground-level urban-design quality relates to passengers’ perceived safety and actual crime rates, subsequently affecting metro ridership. Current literature lacks intensive investigations into how the intertwined physical features and subjective perceptions of micro-scale street environments around subway stations correlate with crime frequencies. It sets out to quantify the correlations between crime reports and urban design quality within the ¼-mile buffer zone of Manhattan subway entrances with the application of Street View Imagery (SVI) and the artificial intelligence of computer vision (CV) and machine learning (ML). Key findings are 1) subjectively and objectively measured urban design quality from SVIs improve explanations of crime. 2) higher perceived safety does not necessarily link with lower crime risks. 3) parks as a point of interest (POI) serve as a crime deterrent. This study has significant implications for urban design and transportation policies and provides references for other urban areas to facilitate safer public transit services and systems by enhancing built environments.KeywordsUrban design qualitySubway entrance crimeStreet View Imagery (SVI)Objective measureHuman perception

Understanding intrinsic patterns and predicting spatiotemporal characteristics of cities require a comprehensive representation of urban neighborhoods. Existing works relied on either inter- or intra-region connectivities to generate neighborhood representations but failed to fully utilize the informative yet heterogeneous data within neighborhoods. In this work, we propose Urban2Vec, an unsupervised multi-modal framework which incorporates both street view imagery and point-of-interest (POI) data to learn neighborhood embeddings. Specifically, we use a convolutional neural network to extract visual features from street view images while preserving geospatial similarity. Furthermore, we model each POI as a bag-of-words containing its category, rating, and review information. Analog to document embedding in natural language processing, we establish the semantic similarity between neighborhood (“document”) and the words from its surrounding POIs in the vector space. By jointly encoding visual, textual, and geospatial information into the neighborhood representation, Urban2Vec can achieve performances better than baseline models and comparable to fully-supervised methods in downstream prediction tasks. Extensive experiments on three U.S. metropolitan areas also demonstrate the model interpretability, generalization capability, and its value in neighborhood similarity analysis.

Street space quality assessment refers to the extraction and appropriate evaluation of the space quality information of urban streets, which is usually employed to improve the quality of urban planning and management. Compared to traditional approaches relying on expert knowledge, the advances of big data collection and analysis technologies provide an alternative for assessing street space more precisely. With street view imagery (SVI), points of interest (POI) and comment data from social media, this study evaluates street space quality from the perspective of exploring and discussing the relationship among street vitality, service facilities and built environment. Firstly, a transfer-learning-based framework is employed for SVI semantic segmentation to quantify the street built environment. Then, we use POI data to identify different urban functions that streets serve, and comment data are utilized to investigate urban vitality composition and integrate it with different urban functions associated with streets. Finally, a function-driven street space quality assessment approach is established. To examine its applicability and performance, the proposed method is experimented with data from part area in Xiamen, China. The output is compared to results based on expert opinion using the correlation analysis method. Results show that the proposed assessment approach designed in this study is in accordance with the validation data, with the overall R2 value being greater than 0.6. In particular, the proposed method shows better performance in scenic land and mixed functional streets with R2 value being greater than 0.8. This method is expected to be an efficient tool for discovering problems and optimizing urban planning and management.

This paper reviews representation learning for geospatial data, focusing on methods for automatically extracting meaningful features from diverse data types. By simplifying tasks and improving accuracy, representation learning has emerged as a powerful tool for geospatial analysis. Due to its generalizability and scalability, representation learning provides an effective approach to processing geospatial data, which is inherently diverse and unstructured. We summarize the representation learning methods for different geospatial data types, including locations, points of interest (POIs), trajectories, spatial interactions, remote sensing imagery, and street view imagery. Treating each data type as a distinct modality, we emphasize the potential of multi-modal representation learning to advance the understanding of geographical phenomena and propose an LLM-guided framework as a potential solution. The review concludes by highlighting the need for further research to improve multi-modal data alignment and enhance the interpretability of feature representations, particularly in complex and dynamic geographical environments.

Social distancing, defined as maintaining a minimum interpersonal distance (often 6 ft or 1.83 m), is a non-pharmaceutical intervention to reduce infectious disease transmission. While numerous quantitative studies have examined people's social distancing behaviors using mobile phone data, large-scale quantitative analyses of adherence to suggested minimum interpersonal distances are lacking. We analyzed pedestrians' social distancing behaviors of using 3 years of street view imagery collected in a metropolitan city (Seattle, WA, USA) during the COVID-19 pandemic. We employed computer vision techniques to locate pedestrians in images, and a geometry-based algorithm to estimate physical distance between them. Our results indicate that social distancing behaviors correlated with key factors such as vaccine availability, seasonality, and local socioeconomic data. We also identified behavioral differences at various points of interest within the city (e.g., parks, schools, faith-based organizations, museums). This work represents a first of its kind longitudinal study of outdoor social distancing behaviors using computer vision. Our findings provide key insights for policymakers to understand and mitigate infectious disease transmission risks in outdoor environments.

Urban network analytics has become an essential tool for understanding and modeling the intricate complexity of cities. We introduce the Urbanity data repository to nurture this growing research field, offering a comprehensive, open spatial network resource spanning 50 major cities in 29 countries worldwide. Our workflow enhances OpenStreetMap networks with 40 + high-resolution indicators from open global sources such as street view imagery, building morphology, urban population, and points of interest, catering to a diverse range of applications across multiple fields. We extract streetscape semantic features from more than four million street view images using computer vision. The dataset’s strength lies in its thorough processing and validation at every stage, ensuring data quality and consistency through automated and manual checks. Accompanying the dataset is an interactive, web-based dashboard we developed which facilitates data access to even non-technical stakeholders. Urbanity aids various GeoAI and city comparative analyses, underscoring the growing importance of urban network analytics research.

Determinants of housing prices are particularly significant for monitoring and understanding housing prices. Traditional variables are measured through official statistics or questionnaire surveys, which are labour intensive and time-consuming. New forms of data, such as point of interest or street view imagery, have been used to extract housing location and neighbourhood features, but they cannot capture how different individuals recognised and evaluated the properties nearby, which may also be relevant in the house price estimation. Therefore, this study investigates whether user-generated images may be used to monitor and understand housing prices and how they influence real estate values. Within this context, perceived scenes features are extracted and quantified to blend with commonly used determinants of housing prices. Two machine learning algorithms, random forest and gradient boosting machines, are utilised and deployed for integration with a typical housing price modelling-hedonic price model. By comparing the performance and interpretability of different models, the relative importance of features and how they influence the estimation power of the models is visualised and analysed. The findings suggest that random forest predictions perform the best and are interpretable, with geotagged Flickr images adding 4.6% to the model’s accuracy (R2) from 61.9% to 66.5%. Although user-generated images increase minor value in house price estimation, they may be used as a supplementary data source to capture perception features for house price estimation. This could help the restructuring and optimisation of residential areas in future regional construction, planning and development.

A unified deep learning framework for urban functional zone extraction based on multi-source heterogeneous data

POI recommendation for occasional groups Based on hybrid graph neural networks

National-scale empirical models of air pollution (e.g., Land Use Regression) rely on predictor variables (e.g., population density, land cover) at different geographic scales. These models typically lack microscale variables (e.g., street level), which may improve prediction with fine-spatial gradients. We developed microscale variables of the urban environment including Point of Interest (POI) data, Google Street View (GSV) imagery, and satellite-based measures of urban form. We developed United States national models for six criteria pollutants (NO2, PM2.5, O3, CO, PM10, SO2) using various modeling approaches: Stepwise Regression + kriging (SW-K), Partial Least Squares + kriging (PLS-K), and Machine Learning + kriging (ML-K). We compared predictor variables (e.g., traditional vs microscale) and emerging modeling approaches (ML-K) to well-established approaches (i.e., traditional variables in a PLS-K or SW-K framework). We found that combined predictor variables (traditional + microscale) in the ML-K models outperformed the well-established approaches (10-fold spatial cross-validation (CV) R2 increased 0.02-0.42 [average: 0.19] among six criteria pollutants). Comparing all model types using microscale variables to models with traditional variables, the performance is similar (average difference of 10-fold spatial CV R2 = 0.05) suggesting microscale variables are a suitable substitute for traditional variables. ML-K and microscale variables show promise for improving national empirical models.

Abstract. Roadside noise barriers (RNBs) are important urban infrastructures to ensure that cities remain liveable. However, the absence of accurate and large-scale geospatial data on RNBs has impeded the increasing progress of rational urban planning, sustainable cities, and healthy environments. To address this problem, this study creates a vectorized RNB dataset in China using street view imagery and a geospatial artificial intelligence framework. First, intensive sampling is performed on the road network of each city based on OpenStreetMap, which is used as the georeference for downloading 6×106 Baidu Street View (BSV) images. Furthermore, considering the prior geographic knowledge contained in street view images, convolutional neural networks incorporating image context information (IC-CNNs) based on an ensemble learning strategy are developed to detect RNBs from the BSV images. The RNB dataset presented by polylines is generated based on the identified RNB locations, with a total length of 2667.02 km in 222 cities. Last, the quality of the RNB dataset is evaluated from two perspectives, i.e., the detection accuracy and the completeness and positional accuracy. Specifically, based on a set of randomly selected samples containing 10 000 BSV images, four quantitative metrics are calculated, with an overall accuracy of 98.61 %, recall of 87.14 %, precision of 76.44 %, and F1 score of 81.44 %. A total length of 254.45 km of roads in different cities are manually surveyed using BSV images to evaluate the mileage deviation and overlap level between the generated and surveyed RNBs. The root mean squared error for the mileage deviation is 0.08 km, and the intersection over union for overlay level is 88.08 % ± 2.95 %. The evaluation results suggest that the generated RNB dataset is of high quality and can be applied as an accurate and reliable dataset for a variety of large-scale urban studies, such as estimating the regional solar photovoltaic potential, developing 3D urban models, and designing rational urban layouts. Besides that, the benchmark dataset of the labeled BSV images can also support more work on RNB detection, such as developing more advanced deep learning algorithms, fine-tuning the existing computer vision models, and analyzing geospatial scenes in BSV. The generated vectorized RNB dataset and the benchmark dataset of labeled BSV imagery are publicly available at https://doi.org/10.11888/Others.tpdc.271914 (Chen, 2021).

Robust tracking with interest points: A sparse representation approach