Street View Images Research Articles

Identification and Evaluation of Representative Places in Cities Using Multisource Data: Focusing on Human Perception

Discovering the Representative places (RPs) of a city will benefit the understanding of local culture and help to improve life experiences. Previous studies have been limited in regard to the large-scale spatial identification of RPs due to the vagueness of boundaries and the lack of appropriate data sources and efficient tools. Furthermore, human perception of these places remains unclear. To address this gap, this research adopts a novel approach to identify and evaluate the RPs of a city from the perspective of human perception. Our methodology involves the utilization of deep learning systems, text semantic analysis, and other techniques to integrate multi-source data, including points of interest (POIs), street view images, and social media data. Taking Nanjing, China, as a case, we identified 192 RPs and their perceptual ranges (PRRPs). The results show the following: (1) Comparing RPs to non-RPs, RPs show higher average scores across four perceptual dimensions (positive indicators): Beautiful (7.11% higher), Lively (34.23% higher), Safety (28.42% higher), and Wealthy (28.26% higher). Conversely, RPs exhibit lower average scores in two perceptual dimensions (negative indicators): Boring (79.04% lower) and Depressing (20.35% lower). (2) Across various perceptual dimensions, RPs have utilized 15.13% of the land area to effectively cover approximately 50% of human perceptual hotspots and cold spots. (3) The RPs exhibit significant variations across different types, levels, and human preferences. These results demonstrate the positive perceived effects that RPs have, providing valuable insights to support urban management, the transformation of the built environment, and the promotion of sustainable urban development, and provide guidance for urban planners and designers to make improvements in urban design and planning to make these sites more attractive.

Multimodal Large Language Models as Built Environment Auditing Tools

This research showcases the transformative potential of large language models (LLMs) for built environment auditing from street-view images. By empirically testing the performances of two multimodal LLMs, ChatGPT and Gemini, we confirmed that LLM-based audits strongly agree with virtual audits processed by a conventional deep learning-based method (DeepLabv3+), which has been widely adopted by existing studies on urban visual analytics. Unlike conventional field or virtual audits that require labor-intensive manual inspection or technical expertise to run computer vision algorithms, our results show that LLMs can offer an intuitive tool despite the user’s level of technical proficiency. This would allow a broader range of policy and planning stakeholders to employ LLM-based built environment auditing instruments for smart urban infrastructure management.

Nonlinear and threshold effects of the built environment, road vehicles and air pollution on urban vitality

The impact of factors such as the built environment, road vehicles, and air quality on urban vitality attracts increasing interest in urban planning and design research. However, tacit assumptions of linear relationships between these factors have been embedded in most studies, leading to biased estimations of their effects on urban vitality. This study addresses the gap by using machine learning models and SHAP (SHapley Additive exPlanations) to investigate the non-linear and threshold effects of the built environment, road vehicles and air pollution on urban vitality, using Manhattan as a study case. Urban vitality was represented by pedestrian presence in 29,540 street-view images. Results showed that Extreme Gradient Boosting outperformed Ordinary Least Squares, Random Forest, and Gradient Boosting Decision Trees in urban vitality estimation. It reveals that while the built environment variables explained a significant portion (77.5 %) of the variance in urban vitality, road vehicles (such as bicycles, buses, cars and motorbikes) and ozone concentrations accounted for 15.18 % and 1.46 %, respectively. The built environment and road vehicle factors exhibit positive nonlinear relationships with urban vitality. Meanwhile, ozone concentration demonstrated a negative threshold effect on urban vitality with a threshold at 27.5 ppb. This study advances our understanding of the threshold effect mechanism of the factors on urban vitality, offering insights into fostering sustainable urban environment.

Assessing the nonlinear impact of green space exposure on psychological stress perception using machine learning and street view images.

Urban green space (GS) exposure is recognized as a nature-based strategy for addressing urban challenges. However, the stress relieving effects and mechanisms of GS exposure are yet to be fully explored. The development of machine learning and street view images offers a method for large-scale measurement and precise empirical analysis. This study focuses on the central area of Shanghai, examining the complex effects of GS exposure on psychological stress perception. By constructing a multidimensional psychological stress perception scale and integrating machine learning algorithms with extensive street view images data, we successfully developed a framework for measuring urban stress perception. Using the scores from the psychological stress perception scale provided by volunteers as labeled data, we predicted the psychological stress perception in Shanghai's central urban area through the Support Vector Machine (SVM) algorithm. Additionally, this study employed the interpretable machine learning model eXtreme Gradient Boosting (XGBoost) algorithm to reveal the nonlinear relationship between GS exposure and residents' psychological stress. Results indicate that the GS exposure in central Shanghai is generally low, with significant spatial heterogeneity. GS exposure has a positive impact on reducing residents' psychological stress. However, this effect has a threshold; when GS exposure exceeds 0.35, its impact on stress perception gradually diminishes. We recommend combining the threshold of stress perception with GS exposure to identify urban spaces, thereby guiding precise strategies for enhancing GS. This research not only demonstrates the complex mitigating effect of GS exposure on psychological stress perception but also emphasizes the importance of considering the "dose-effect" of it in urban planning and construction. Based on open-source data, the framework and methods developed in this study have the potential to be applied in different urban environments, thus providing more comprehensive support for future urban planning.

Past, Present, and Future Perspectives on the Integration of AI Into Walkability Assessment Tools: A Systematic Review

This study employs a systematic literature review (PRISMA methodology) to investigate the integration of Artificial Intelligence (AI) in walkability assessments conducted between 2012 and 2022. Analyzing 34 articles exploring data types, factors, and AI tools, the review emphasizes the value of utilizing diverse datasets, particularly street view images, to train supersized AI models. This approach fosters efficient, unbiased assessments and offers deep insights into pedestrian environment interactions. Furthermore, AI tools empower walkability assessment by facilitating mapping, scoring, designing pedestrian routes, and uncovering previously unconsidered factors. The current shift from large-scale spatial data analysis (allocentric perspective) to a ground-level view (egocentric perspective) and physical and perceptual features of walking introduces a subjective lens into current walkability assessment tools. However, the efficacy of current methods in addressing non-visual aspects of human perception and their applicability across diverse demographics remains debatable. Finally, the lack of integration of emerging technologies like virtual/augmented reality and digital twin leaves a significant gap in research, inviting further study to determine their efficacy in enhancing the current methods and, in general, understanding the interaction of humans and cities.

Combining human perception and street accessibility to provide information for better street construction: a case study of Chengdu City, China

ABSTRACT High-quality street spaces have a positive impact on residents’ lives. Revealing the differences in human perception and walkability across street spaces can highlight priority areas for street quality enhancement and provide valuable insights that help urban managers make informed decisions. This study first utilised machine learning models (SegNet) to extract visual elements from street view images and collected volunteers’ ratings of six perceptual dimensions of street scenes. Subsequently, it employed random forest algorithms and spatial autocorrelation models to predict and analyse overall street perception. Furthermore, principal component analysis was used to consolidate the six perceptual scores, coupled with an analysis of street walkability, to identify streets that most required quality enhancement. Finally, correlation and regression analyses were conducted between human perception ratings and street visual elements. The results indicated that buildings positively influence perceptions of wealth, safety, liveliness, and beauty and are the most significant positive perception elements. Conversely, walls were the most significant negative perception elements. Additionally, trees positively impacted perceptions of liveliness and beauty. Lastly, streets categorised as “high walkable-low perception” exhibited a radial dispersion from the city centre. In conclusion, the findings of this study provide valuable insights for enhancing street-space quality in Chengdu and other cities.

The mediating role of visual behavior in perceived walkability evaluation

ABSTRACT Past studies used street-view images to directly analyze the effect of visual environments on perceived walkability. There is a gap in understanding how eye movement mediates the results of perceived walkability evaluations. The study proposed a method to transform the vector coordinates of gaze into the coordinates of panoramic photos. Then, we used mixed linear regressions and a simple mediation model to reveal the mediating role of visual behavior on perceived walkability evaluations. The results showed that: (1) Fixation has an incomplete mediating role in the impact of visual information on perceived walkability. (2) Most streetscape elements with long fixation duration impacted perceived walkability. (3) In the visual information model, the tree and streetlight negatively impacted perceived walkability. And the gutter positively impacted perceived walkability. However, in the fixation model, the tree and streetlight positively impacted perceived walkability. The gutter negatively impacted perceived walkability. This study offers scholars and policymakers a novel perspective on understanding perceived walkability.

Virtual Assessment of Physical Activity-Related Built Environment in Soweto, South Africa: What Is the Role of Contextual Familiarity?

Understanding how urban environments shape physical activity is critical in rapidly urbanizing countries such as South Africa. We assessed the reliability of virtual audits for characterizing urban features related to physical activity in Soweto, South Africa. We used the Microscale Audit of Pedestrian Streetscapes Global tool to characterize pedestrian-related features from Google Street View images in four neighborhoods of Soweto. Neighborhoods were selected to represent different levels of deprivation. Inter-rater reliability was analyzed according to the rater's familiarity with the local area. The results show a higher inter-rater reliability was observed among auditors with greater contextual familiarity. Many measurements however generated inconclusive results due to either low variability in the raters' responses or the absence of the features in the streets. It is evident from our findings that virtual audits are efficient tools that can be used to assess the built environment. However, to ensure meaningful use of these tools in diverse settings, we recommend that auditors comprise of people with contextual familiarity.

Urban energy management system based on intelligent linker

Introduction: The use of machine learning (ML) approaches to improve energy utilization in smart urban environments has garnered significant attention in recent years.Objective: This research presents an innovative structure called a bi-fold mechanism-driven convolutional deep network (BMCDN) for estimating the energy performance of urban public facilities in urban energy management systems.Methods: The suggested method includes two significant phases: (1) feature extraction and fusion, and (2) energy significance estimation. The attention fine-tuned ResNet (N1) processes street-view images to evaluate anticipated market significance levels, while the attention-based Bi-LSTM (N2) integrates cross-domain features using input attention. A decision tree (DT) is used to combine and evaluate the fused information and estimated values, serving as the energy value estimator to determine energy values. Data gathered related to public facilities' energy efficiency from various sources is used to analyze the effectiveness of the suggested framework. Results: The research presents an analysis of the performance gains using image-only representations and a proposed approach with morphological traits. The findings demonstrate that incorporating smart urban-related façade images improves the accuracy of the proposed framework and highlights the connection between energy usage and public facilities.Conclusions: This study shows the potential for significant precision along with rapid inference time in predicting the energy performance of urban public facilities by combining data from numerous sources. Keywords: Urban energy management; public facilities; machine learning (ML); multi-source data; bi-fold mechanism-driven convolutional deep network (BMCDN)

SAGE-GSAN: A graph-based method for estimating urban taxi CO emissions using street view images

Accurately predicting the carbon emissions of urban street vehicles is a current challenge in the field of urban transportation. This study proposed a new SAGE-GSAN model (Graph SAmple and aggreGatE - Graph Spatial Attention Network) to solve this problem. It combines graph neural networks with streetscape features and road network structure to predict the traffic carbon monoxide emissions at the street level. The method takes the street view images, the 5075 roads network structure in Wuhan and 19 street visual elements as the input features, and the carbon monoxide emissions obtained from the driving trajectories as the prediction data. The method achieved an experimental accuracy of 81.4% in predicting carbon monoxide emissions from street cabs. This study also compares the prediction results of traditional neural networks and analyzes the effects of different street-level features and graph convolution layers on the prediction accuracy. The results of this study show that the graph neural network and attention mechanism techniques could solve the fine-grained carbon emission prediction problem at the urban street level effectively. The model code is shared at the: https://github.com/zou9229/CO_Predict_Code.

The effects of built environments on bicycle accidents around bike-sharing program stations using street view images and deep learning techniques: The moderating role of streetscape features

With the global rise of bike-sharing programs (BSP), planners and traffic experts have raised concerns as to whether the rapid growth of BSP ensures cycling safety. Despite numerous studies on built environments encouraging bike usage, there is limited knowledge whether streetscape environments around BSP stations affect bicycle accidents. We address this gap by investigating the relationships between various built environments and bicycle accidents around BSP stations, with a particular focus on the moderating effects of comprehensive streetscapes. Streetscape environments, estimated through semantic segmentation and k-means clustering, were used in two-level negative binomial regression models to clarify how street- and station-level environments affect different types of bicycle accidents. The findings indicate increased crash likelihoods on higher-speed roads, streets with traffic facilities, in proximity to public transportation infrastructure, and specific streetscape types. In particular, streetscape features like green oasis streets and open-sky roadways positively contribute to mitigating the negative effects of traffic facilities, such as bike networks and crosswalks, on cycling safety. This study aids in developing comprehensive strategies and guidelines to retrofit built environments for better traffic safety and thereby promote urban cycling.

Learning Building Floor Numbers from Crowdsourced Streetview Images

Learning Building Floor Numbers from Crowdsourced Streetview Images

Measuring shaded bike lanes for heat stress mitigation with deep learning: A case study in Amsterdam, Netherlands

As global warming intensifies and heatwaves become more frequent, thermal comfort has emerged as a crucial factor in creating habitable and equitable urban environments. Cities worldwide are promoting sustainable transportation and reducing vehicle usage by enhancing cycling infrastructures. However, the thermal comfort of cyclists has often been neglected, with most research focusing on pedestrians. This study introduces a framework for evaluating shade levels along the bike lanes by incorporating a ‘Shade Index’ that quantifies street shadiness. Utilizing deep learning techniques, the index analyzes street view images from various bike lane locations to evaluate the contribution of trees and buildings in providing shade. The application of this framework is a shade rating map of the bike lanes in Amsterdam, providing cyclists with alternatives that prioritize coolness over the shortest routes typically suggested by map applications, particularly useful during warmer weather. The study identifies significant variability in shade provision, with central urban areas typically more shaded than outlying regions. The workflow can be adapted to other cities to enhance shade provision for cycling infrastructures.

Mapping sense of place as a measurable urban identity: Using street view images and machine learning to identify building façade materials

Sense of place, as an intangible perception, is widely recognized as an urban identity and of great value in both cross-cultural studies and contemporary urbanism. Building façade material can effectively capture sense of place due to its combination of physical and social attributes. Nevertheless, there are no widely implementable and high-resolution approaches to identify façade materials on a large scale. As a response, this study proposes a method using street view images (SVIs) and a set of deep Convolutional Neural Networks (CNNs) to identify building façade materials. Specifically, a large cross-cultural training set was built to promote generalizability. Buildings within SVIs were divided into high-resolution rectangular images and classified using a well-trained Residual Network-50 (ResNet-50) model. Sense of place and its spatial patterns were then depicted by measuring façade material and analytical indicators including diversity and continuity. Eight cities worldwide with distinctive urban identities were examined. The findings revealed that compared to Asian cities, New York City, Chicago, and London are similar, while Paris and Tokyo are more distinctive. While challenges persist in comprehensively measuring the sense of place, the analysis of façade materials offers an insightful indicator that can assist in enhancing urban identity for contemporary urbanism. This study not only promotes the fine development of urban science through the empowerment of intelligent algorithms but also introduces a new perspective on exploring unmeasurable qualities based on the objective physical environment.

Urban streets profiling with coupled spatio-temporal characteristics and topological information from the biking perspective

Urban street profiling is the spatio-temporal pattern discovery of street-level urban areas, which plays a vital role in understanding urban structures and dynamics. Due to the natural topology and various geographic characteristics on the streets, it is necessary to combine multi-dimensional spatio-temporal information to understand different profiles of streets. This research aims to develop a street profiling framework according to the coupled characteristics of streets. At the start, a bidirected dual graph and a spatial weighted graph embedding method were used to solve the street representation. Then, the street profiles can be extracted by clustering embedding vectors of streets and feature importance analysis. As the case study, we employed the bike trajectories and street view images in Xiamen, China to depict the geographic attributes of streets. The results can reveal nine spatio-temporal street profiles from the biking perspective, including three spatial distribution patterns and two spatial semantic patterns. Urban streets in the study area show a significant hierarchical pattern because of locations and the spatial lags of the biking behaviors. Meanwhile, the spatio-temporal characteristics of biking behaviors are the main factors of street profiles, though the street environment attributes participate in over half the number of profile types. We further evaluated the profiling ability of the proposed framework and the importance of urban street profiles using coupled characteristics. Overall, this study explored the profiling method for coupling static and dynamic characteristics of urban streets. The profiling results also help understand street usage and experiences by bikers, which have a practical value on the human-oriented classification of streets and further urban development from a geographic view.

Understanding urban traffic flows in response to COVID-19 pandemic with emerging urban big data in Glasgow

Urban traffic analysis has played an important role in urban development, providing insights for urban planning, traffic management, and resource allocation. Meanwhile, the global pandemic of COVID-19 has significantly changed people's travel behaviour in urban areas. This research uses the spatial Durbin model to understand the relationship between traffic flows, urban infrastructure, and socio-demographic indicators before, during, and after pandemic periods. We include factors such as road characteristics, socio-demographics, surrounding built environments (land use and nearby points of interest), and the emerging urban big data source of Google Street View images to understand their influences on time series traffic flows. Taking the city of Glasgow as the case study, we have found that areas with more young and white dwellers are associated with more traffic flows, while natural green spaces are associated with fewer traffic flows. Major roads between cities and towns also show heavier traffic flows. Besides, the application of Google Street View images in this research has revealed the heterogeneous effects of green space on urban traffic flows, as the magnitudes of their effects vary by distance. We also detect that the spatial dependence between adjacent neighbourhoods among the traffic flows and associated urban parameters is variable during the four COVID-19 periods. With the influence of COVID-19, there has been a significant decrease in long-distance travel. The noticeable change in travel behaviour presents a valuable opportunity to encourage active travel in the near future.

Multi‐view street view image fusion for city‐scale assessment of wind damage to building clusters

Multi‐view street view image fusion for city‐scale assessment of wind damage to building clusters

UB-FineNet: Urban building fine-grained classification network for open-access satellite images

Fine classification of city-scale buildings using satellite imagery is a crucial research area with significant implications for urban planning, infrastructure development, and population distribution analysis. However, the task faces great challenges due to low-resolution overhead images acquired from high-altitude space-borne platforms and the long-tailed sample distribution of fine-grained urban building categories, leading to a severe class imbalance problem. To address these issues, we propose a deep network approach to the fine-grained classification of urban buildings using open-access satellite images. A Denoising Diffusion Probabilistic Model (DDPM) based super-resolution method is first introduced to enhance the spatial resolution of satellite images, which benefits from domain-adaptive knowledge distillation. Then, a new fine-grained classification network with Category Information Balancing Module (CIBM) and Contrastive Supervision (CS) technique is proposed to mitigate the problem of class imbalance and improve the classification robustness and accuracy. Experiments on Hong Kong data set with 11 distinct building types revealed promising classification results with a mean Top-1 accuracy of 60.45%, which is on par with street-view image based approaches. A comprehensive ablation study demonstrates that the CIBM and CS modules improve Top-1 accuracy by 2.6% and 3.5%, respectively, over the baseline approach. In addition, these modules can be easily integrated into other classification networks, achieving similar performance improvements. This research advances urban analysis by providing an effective solution for detailed classification of buildings in complex mega-city environments using only open-access satellite imagery. The proposed technique can serve as a valuable tool for urban planners, aiding in the understanding of economic, industrial, and population distribution within cities and regions, ultimately facilitating informed decision-making in urban development and infrastructure planning. Data and code will be publicly available at https://github.com/ZhiyiHe1997/UB-FineNet.

Spatial heterogeneity and nonlinearity study of bike-sharing to subway connections from the perspective of built environment

Bike-sharing has become an effective solution to address the "last mile" challenge in public transit. Existing studies insufficiently consider architectural environmental factors from a design perspective in impacting bike-sharing-to-subway connection(BSC) behavior, and often relying on subjective assessments to explain the spatial heterogeneity of the built environment on BSC travel. To investigate the impact mechanisms of the built environment on BSC behaviors, this study used weekly data on bike-sharing travel trajectories from April 7 to April 14, 2021, in Shenzhen, China, as the research sample. By integrating large-scale urban point of interest(POI), street-view images and road-network data from Shenzhen, this study examines the mechanisms by which the built environment influences BSC travel. This study initially applied linear regression and multiscale geographically weighted regression (MGWR) to analyze the spatial heterogeneity of built environment impacts on BSC travel. Then, XGBoost's nonlinear relationships were used to explain and validate the MGWR results. This study reveals that functional diversity is the most significant factor influencing BSC travel. At subway stations where it is difficult to enhance functional diversity, improvements in road connectivity, vegetation, and enclosure can enhance BSC commuting. These findings are crucial for decreasing the reliance on motor vehicles and fostering sustainable urban growth.

Sensing noise exposure and its inequality based on noise complaint data through vision-language hybrid method

This study seeks to reveal urban noise exposure patterns and inequalities using noise complaint data and vision-language hybrid method. By applying a natural language processing model to 17,243 noise complaint records, we uncovered distinct patterns of traffic, industrial, and living noise exposures across residential communities. Our analysis of street view images near complaint locations, utilizing a Residual Network (ResNet) model and Class Activation Mapping (CAM), identified the key environmental elements of different noise sources. Notably, our assessment of noise exposure inequality across 9791 communities yielded a counterintuitive finding: contrary to previous studies in Western contexts, rich communities in China experience higher and more unequal noise exposure compared to average communities, with Gini coefficients exceeding 0.8. This unexpected result likely stems from China's unique rapid urbanization process. Our use of crowdsourced complaint data aligns more closely with human subjective perceptions of noise, offering a novel perspective on noise exposure inequality. These findings challenge existing assumptions about the relationship between socioeconomic status and environmental quality in urban China, and have significant implications for urban planning and noise management strategies in rapidly developing cities.