Street View Images Research Articles

Extracting the pickpocketing information implied in the built environment by treating it as the anomalies

The practice of crime risk mapping, enabled by the utilization of geospatial big data such as street view images, has received significant research attention. However, in situations where available data is scarce, mapping models may suffer from underfitting and generate inaccurate spatial pattern estimations of crime risk. The covert nature of pickpocketing crimes results in limited observed areas relevant to such criminal events, leading to insufficient coverage of geospatial data. Moreover, the location of crime is also influenced by socio-economic characteristics that may introduce biases into crime risk estimates. These factors render it challenging for the model to capture a valid crime risk pattern, potentially yielding misleading conclusions. Therefore, effectively extracting crime risk with limited data remains a challenge, especially when relying on easily accessible, widespread, and unbiased geospatial data. To address this challenge, we propose a novel crime risk assessment framework based on deep anomaly detection techniques, assuming that urban landscape anomalies carry deep crime risk information. We take Shenzhen as the study area and map the distribution of pickpocketing risk using street view images, accurately revealing the spatial aggregation of pickpocketing crime risk. Our findings indicate that pickpocketing crime in China is caused by regional economic conditions, built environment factors, and human routine activities. This study provides valuable insights for policing and prevention strategies aimed at addressing pickpocketing crimes in large Chinese cities. By leveraging our proposed crime risk assessment framework, decision-makers can allocate resources more efficiently and develop targeted interventions to mitigate crime risks.

Effects of built environment on metro ridership at a microscopic scale: a case study of Xi’an, China

ABSTRACT Few studies have examined the relationship at the microscopic spatial scale. In this study, multiple sources of data including mobile phone signal data, automatic fare collection system data, geo-information data, and street-view image data are combined to measure metro ridership and built environment at the plot or block scale. The Random Gradient Boosting Decision Tree was used to explore relationship between the built environment and ridership. The results show the following: (1) the relationship between built environment and ridership shows different types of curves. (2) The path distance to the metro station and the visual perception of road space have more significant impacts on ridership than road network density. (3) The location of the grid also affects grid-level metro ridership. The results suggest that planners should consider the locational factors, pay attention to the different effective thresholds of different variables on ridership and the longitudinal landscaping of non-motorized urban roads.

UPDExplainer: An interpretable transformer-based framework for urban physical disorder detection using street view imagery

Urban Physical Disorder (UPD), such as old or abandoned buildings, broken sidewalks, litter, and graffiti, has a negative impact on residents’ quality of life. They can also increase crime rates, cause social disorder, and pose a public health risk. Currently, there is a lack of efficient and reliable methods for detecting and understanding UPD. To bridge this gap, we propose UPDExplainer, an interpretable transformer-based framework for UPD detection. We first develop a UPD detection model based on the Swin Transformer architecture, which leverages readily accessible street view images to learn discriminative representations. To provide clear and comprehensible evidence and analysis, we subsequently introduce a UPD factor identification and ranking module that combines visual explanation maps with semantic segmentation maps. This novel integrated approach enables us to identify the exact objects within street view images that are responsible for physical disorders and gain insight into the underlying causes. Experimental results on the re-annotated Place Pulse 2.0 dataset demonstrate promising detection performance of the proposed method, surpassing the state-of-the-art with an increase of 6.54% in Accuracy and 7.05% in F1 score. For a comprehensive evaluation of the method’s ranking performance, we provide both quantitative and qualitative results, highlighting the model’s precise identification of one, two, three, and four UPD factors, respectively. We also present a case study of detecting and ranking physical disorders in the southern region of downtown Los Angeles, California, to demonstrate the practicality and effectiveness of our framework.

Measuring Objective Walkability from Pedestrian-Level Visual Perception Using Machine Learning and GSV in Khulna, Bangladesh

Walkability entails measuring the degree of walking activity, a non-motorized mode of active transportation crucial in fast-developing urban settings and combating sedentary lifestyles. While there has been extensive objective research focusing on factors related to the physical environment that influence walkability, there has been a comparatively limited exploration into objectively evaluating a pedestrian’s visual perception. This study in Khulna, Bangladesh, aimed to develop a novel method for objectively measuring walkability based on pedestrian-level visual perception using machine learning. In this research, ResNet, a computer vision model, analyzed 127 panoramic Google Street View images taken at 200-meter intervals from seven major roads. The model, trained with the “deeplabv3plusResnet18CamVid” algorithm, quantified five selected visual features. The results, including walkability rankings, correlation analysis, and spatial mapping, highlighted that greenery and visual enclosures significantly influenced the walkability index. However, the impact of other visual features was less distinctive due to an overall poor streetscape condition. This study bridged the gap between human perception and scientific intelligence, allowing for the evaluation of previously “unmeasurable” streetscape designs. It provides valuable insights for more human-centered planning and transportation strategies, addressing the challenges of modern urbanization and sedentary behavior.

High or low? Exploring the restorative effects of visual levels on campus spaces using machine learning and street view imagery

According to the Attentional Restoration Theory (ART), cognitive restoration (e.g., Fascination) may occur when the physical environment exhibits high restorative quality. However, these studies usually ignore the effect of different levels of visual features on restoration quality, and small-scale questionnaires are difficult to use to comprehensively evaluate the restoration quality of a space. In this study, we propose a machine learning based method for high-resolution, large-scale assessment of the restoration quality of campus environments using Street View Images (SVIs). First, visual features are extracted from campus SVIs using computer vision method. Second, an online survey using the PRS-11 questionnaire (containing four indicators: Being-away, Coherence, Scope, and Fascination) was conducted to label the images. Finally, we developed a regression model to predict campus restorative quality and to model the non-linear relationship between the visual features of SVIs and this quality. We studied 1088 SVIs in the Lihu campus of Jiangnan University (JNU) to verify the feasibility of our method, and the results showed that SVIs can accurately help us predict the restoration quality of the campus environment on a large scale (R2 = 0.726). Next, we examined the variance in visual features between campus spaces with different levels of restorative quality, and investigated the effect of different levels of visual features on restoration quality. We found that contributions of high-level visual features to restoration, such as trees, are robust (Adj R2 = 0.504) compared to low-level visual features (Adj R2 = 0.032) that included such as color information. This provides a new perspective for assessing recovery environments and designing healthy campus environments. The code is shared at: https://github.com/MMHHRR/Restorative_Quality

Uncovering the association between traffic crashes and street-level built-environment features using street view images

Investigating the relationship between built environment factors and roadway safety is crucial for preventing road traffic accidents. Although studies have analyzed traffic-related built environment factors based on pre-determined zonal units, conclusive evidence regarding the relationship between streetscape features and traffic accidents at a fine-grained road segment level is still lacking. With the widespread availability of large-scale street view images, automatically analyzing urban built environments on a large scale is possible. Therefore, the aim of this study was to investigate the relationship between streetscape features and traffic accidents at a fine-grained road segment level using street view images. Specifically, we employed semantic image segmentation to extract streetscape elements from urban street view images, and then created traffic crash-related variables, including the street-level built environment variables, traffic variables, land-use indices, and proximity characteristics, at the road-segment level. Finally, we adopted a classification-then-regression strategy to model the number of traffic crashes while considering the zero-inflated and spatial heterogeneity issues. Our findings suggest that streetscape features can effectively reflect built-environment characteristics at the road-segment level. Moreover, a comparison of our proposed modeling method with existing models demonstrates its superior performance. The results provide insight into the development of effective planning strategies to improve traffic safety.

Identifying the integrated visual characteristics of greenway landscape: A focus on human perception

Understanding the integrated landscape characteristics and spatial sequences of greenways and the surrounding visual environments is crucial for connecting fragmented natural spaces within cities and enhancing residents’ natural experiences. However, previous studies have mostly predominantly focused on individual perceptions of greenway landscapes, rather than considering the integrated visual characteristics of both the inner and the visible surroundings. This disparity often deviates from people's actual perception of these spaces. To address this gap, we have adopted a new approach to identify the integrated greenway visual characteristics in inner and surrounding landscapes from a human perception perspective, in the city of Wuhan, China. Our methodology involved the utilization of deep learning systems, text semantic analysis, and other techniques to integrate multi-source data, including street view images, remote sensing images, and perceptual comments, to quantitatively assess the landscape sequences and naturalness of landscape changes along the greenway paths. We identify the positions and characteristics of unique landscape nodes within the greenway. The findings of our study revealed ten main landscape types of greenways in Wuhan, with over 72 % of the greenways comprising more than five types. As the length of the greenways increases, the number of landscape types initially experiences rapid growth before reaching a stable state. Furthermore, the frequency of landscape change exhibits a linear and positive correlation with greenway length. Notably, the naturalness of the majority of greenway inner landscapes closely aligns with that of the surrounding visible environment. The openness of all greenways was positively correlated with the proportion of farmland and grassland in their surrounding areas. Furthermore, people exhibit higher positive sentiment responses to greenways that maintain a high level of natural consistency between the inside and outside environments. Additionally, over 47 % of the unique nodes primarily coincide with a specific landscape type and play a vital role in shaping the rhythm of greenway landscape change. These findings provide valuable insights to support the construction and management of greenway landscapes, as well as guidance for urban planners and designers aiming to harness natural resources effectively in the development of urban greenways.

Streetscapes as part of servicescapes: Can walkable streetscapes make local businesses more attractive?

Attractive local businesses can make cities more walkable by providing desirable destinations to walk to. The term servicescape has been used to describe the physical settings and environments that affect customers' inference of the service quality of businesses at that location. This study extends the concept of servicescapes to include walkable streetscapes and examines whether features that make streets more walkable also make local businesses on those streets more attractive. This study measures walkable streetscape features using street view images and computer vision, which are associated with customer satisfaction values derived from Yelp review scores of restaurants in Atlanta, GA, USA. After controlling for various covariates including pedestrian accessibility, restaurant type, and neighborhood-specific characteristics, this study found sidewalk buffers, greenness at eye level, and building-to-street ratio as streetscape features positively associated with customer satisfaction. Planning tools for promoting walkable streetscapes are discussed to improve the street vibrancy and the economic opportunities of local businesses.

Bridging the gap between pedestrian and street views for human-centric environment measurement: A GIS-based 3D virtual environment

Street View Image (SVI) data has gained popularity as a methodological tool for assessing pedestrian-level environments across various disciplines. Several studies in transportation and public health fields posit that environmental features measured through SVIs are similar to those perceived by pedestrians, whereas, in fact, SVIs are typically collected from vehicles. As a result, the SVI data may reflect significant differences from an actual pedestrian’s view. Hence, this study investigates the errors in measuring neighborhood environmental features resulting from the difference between pedestrian and vehicle views. To accomplish this goal, a three-dimensional (3D) urban model is developed, utilizing multiple Geographic Information Systems (GIS) data. This model accurately represents the spatial positioning and dimensions of various entities. From the 3D model, virtual SVIs for each perspective are collected. Subsequently, we quantify environmental features in a manner similar to semantic segmentation for the images. Our findings reveal significant measurement errors for sidewalks, greenery, and roads between pedestrian and vehicle views. These errors are more pronounced with certain environment characteristics and parameter settings for image acquisition. For example, narrow road segments with few on-street parked vehicles exhibit increased measurement errors for greenery. Overall, the study provides guidelines for researchers on utilizing SVI to achieve reliable measurement of environmental features and move toward human-centric measurements.

The Influence of Visual Landscapes on Road Traffic Safety: An Assessment Using Remote Sensing and Deep Learning

Rapid global economic development, population growth, and increased motorization have resulted in significant issues in urban traffic safety. This study explores the intrinsic connections between road environments and driving safety by integrating multiple visual landscape elements. High-resolution remote sensing and street-view images were used as primary data sources to obtain the visual landscape features of an urban expressway. Deep learning semantic segmentation was employed to calculate visual landscape features, and a trend surface fitting model of road landscape features and driver fatigue was established based on experimental data from 30 drivers who completed driving tasks in random order. There were significant spatial variations in the visual landscape of the expressway from the city center to the urban periphery. Heart rate values fluctuated within a range of 0.2% with every 10% change in driving speed and landscape complexity. Specifically, as landscape complexity changed between 5.28 and 8.30, the heart rate fluctuated between 91 and 96. This suggests that a higher degree of landscape richness effectively mitigates increases in driver fatigue and exerts a positive impact on traffic safety. This study provides a reference for quantitative assessment research that combines urban road landscape features and traffic safety using multiple data sources. It may guide the implementation of traffic safety measures during road planning and construction.

Mapping sidewalks on a neighborhood scale from street view images

Although reliable and accurate inventorying of sidewalks is time consuming, it can aid urban planners in decision making for infrastructure development. Recent advancements in computer vision and machine learning algorithms have improved the reliability and accuracy of automated inventorying. This research uses a deep learning architecture-based semantic segmentation model (i.e., HRNet + OCR) to automate sidewalk identification using Google Street View (GSV) images. The results show that retraining the model using local training images yields 114.16% and 178.11% higher performance in terms of intersection over union (IoU) metric compared to pretrained model using Cityscapes and Mapillary datasets, respectively. The developed model showed excellent performance in predicting the presence of sidewalks in an image by achieving high accuracy (0.9557), precision (0.9447), recall (0.9900), and F1- score (0.9668). Further, in comparison with EfficientNet, a computationally efficient image classification model, the present model showed superior performance in predicting sidewalk presence at the image level. Therefore, integrating local training images containing minimum required labels (in this study, roads, sidewalks, buildings, and walls) with publicly available training datasets can help increase the performance of the semantic segmentation model for extracting the required features (in this study, roads and sidewalks) from GSV images, especially in developing countries like Bangladesh. This study generates sidewalk maps on a neighborhood scale, which can be useful visualization tools for researchers and practitioners to understand the existing pedestrian infrastructure and plan for future improvements.

Effect Modifications of Overhead-View and Eye-Level Urban Greenery on Heat-Mortality Associations: Small-Area Analyses Using Case Time Series Design and Different Greenery Measurements.

The protective effect of urban greenery from adverse heat impacts remains inconclusive. Existing inconsistent findings could be attributed to the different estimation techniques used. We investigated how effect modifications of urban greenery on heat-mortality associations vary when using different greenery measurements reflecting overhead-view and eye-level urban greenery. We collected meteorological and daily mortality data for 286 territory planning units between 2005 and 2018 in Hong Kong. Three greenery measurements were extracted for each unit: a) the normalized difference vegetation index (NDVI) from Landsat remote sensing images, b) the percentage of greenspace based on land use data, and c) eye-level street greenery from street view images via a deep learning technique. Time-series analyses were performed using the case time series design with a linear interaction between the temperature term and each of the three greenery measurements. Effect modifications were also estimated for different age groups, sex categories, and cause-specific diseases. Higher mortality risks were associated with both moderate and extreme heat, with relative risks (RRs) of 1.022 (95% CI: 1.000, 1.044) and 1.045 (95% CI: 1.013, 1.079) at the 90th and 99th percentiles of temperatures relative to the minimum mortality temperature (MMT). Lower RRs were observed in greener areas whichever of the three greenery measurements was used, but the disparity of RRs between areas with low and high levels of urban greenery was more apparent when using eye-level street greenery as the index at high temperatures (99th percentile relative to MMT), with RRs for low and high levels of greenery, respectively, of 1.096 (95% CI: 1.035, 1.161) and 0.985 (95% CI: 0.920, 1.055) for NDVI (), 1.068 (95% CI: 1.021, 1.117) and 0.990 (95% CI: 0.906, 1.081) for the percentage of greenspace (), and 1.103 (95% CI: 1.034, 1.177) and 0.943 (95% CI: 0.841, 1.057) for eye-level street greenery (). Health discrepancies remained for nonaccidental mortality and cardiorespiratory diseases and were more apparent for older adults ( years of age) and females. This study provides new evidence that eye-level street greenery shows stronger associations with reduced heat-mortality risks compared with overhead-view greenery based on NDVI and percentage of greenspace. The effect modification of urban greenery tends to be amplified as temperatures rise and are more apparent in older adults and females. Heat mitigation strategies and health interventions, in particular with regard to accessible and visible greenery, are needed for helping heat-sensitive subpopulation groups in coping with extreme heat. https://doi.org/10.1289/EHP12589.

Sensing Accident-Prone Features in Urban Scenes for Proactive Driving and Accident Prevention

In urban cities, visual information on and along roadways is likely to distract drivers and lead to missing traffic signs and other accident-prone (AP) features. To avoid accidents due to missing these visual cues, this paper proposes a visual notification of AP-features to drivers based on real-time images obtained via dashcam. For this purpose, Google Street View images around accident hotspots (areas of dense accident occurrence) identified by a real-accident dataset are used to train a novel attention module to classify a given urban scene into an accident hotspot or a non-hotspot (area of sparse accident occurrence). The proposed module leverages channel, point, and spatial-wise attention learning on top of different CNN backbones. This leads to better classification results and more certain AP-features with better contextual knowledge when compared with CNN backbones alone. Our proposed module achieves up to 92% classification accuracy. The capability of detecting AP-features by the proposed model were analyzed by a comparative study of three different class activation map (CAM) methods, which are used to inspect specific AP-features causing the classification decision. The outputs of the CAM methods were processed by an image processing pipeline to extract only the AP-features that are explainable to drivers and notified using a visual notification system. Range of experiments was performed to prove the efficacy and AP-features of the system. Ablation of the AP-features taking 9.61%, on average, of the total area in each image sample increased the chance of a given area to be classified as a non-hotspot by up to 21.8%.

Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method

Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method

Development of a synthetic dataset generation method for deep learning of real urban landscapes using a 3D model of a non-existing realistic city

In the urban landscaping field, training datasets for instance segmentation in the detection of building facades are needed for complex analysis and simulation based on data. Manual dataset generation methods are costly, so automatic generation is preferred. However, previous methods are costly and are more reliant on 3D real-city models than LOD2. The objective of this research is to propose a method for automatically generating synthetic datasets for instance segmentation using an image translation technique and procedural modeling. The image translation technique is used to generate street view images for training data, and procedural modeling can automatically specify the composition of generated street view images and generate annotation data, as well as a variety of 3D models. The proposed method generates various kinds of synthetic data at a rate of 4.45 s per set, and the model trained on the generated dataset can detect real buildings for each instance.

Data-Driven Approach for Urban Micromobility Enhancement through Safety Mapping and Intelligent Route Planning

Micromobility responds to urban transport challenges by reducing emissions, mitigating traffic, and improving accessibility. Nevertheless, the safety of micromobility users, particularly cyclists, remains a concern in urban environments. This study aims to construct a safety map and a risk-averse routing system for micromobility users in diverse urban environments, as exemplified by a case study in Lisbon. A data-driven methodology uses object detection algorithms and image segmentation techniques to identify potential risk factors on cycling routes from Google Street View images. The ‘Bikeable’ Multilayer Perceptron neural network measures these risks, assigning safety scores to each image. The method analyzed 5321 points across 24 parishes in Lisbon, with an average safety score of 4.5, indicating a generally safe environment for cyclists. Carnide emerged as the safest area, while Alcântara exhibited a higher level of potential risks. Additionally, an equation is proposed to compute route efficiency, enabling comparisons between different routes for identical origin-destination pairs. Preliminary findings suggest that the presented routing solution exhibits higher efficiency than the commercial routing benchmark. Risk-averse routes did not result in a substantial rise in travel distance or time, with increments of 7% on average. The study also contributed to increasing the existing amount of cycle path data in Lisbon by 12%, correcting inaccuracies, and updating the network in OpenStreetMap, providing access to more precise information and, consequently, more routes. The key contributions of this study, such as the safety map and risk-averse router, underscore the potential of data-driven tools for boosting urban micromobility. The solutions proposed demonstrate modularity and adaptability, making them fit for a range of urban scenarios and highlighting their value for cities prioritizing safe, sustainable urban mobility.

Geospatial mapping of distribution grid with machine learning and publicly-accessible multi-modal data

Detailed and location-aware distribution grid information is a prerequisite for various power system applications such as renewable energy integration, wildfire risk assessment, and infrastructure planning. However, a generalizable and scalable approach to obtain such information is still lacking. In this work, we develop a machine-learning-based framework to map both overhead and underground distribution grids using widely-available multi-modal data including street view images, road networks, and building maps. Benchmarked against the utility-owned distribution grid map in California, our framework achieves > 80% precision and recall on average in the geospatial mapping of grids. The framework developed with the California data can be transferred to Sub-Saharan Africa and maintain the same level of precision without fine-tuning, demonstrating its generalizability. Furthermore, our framework achieves a R2 of 0.63 in measuring the fraction of underground power lines at the aggregate level for estimating grid exposure to wildfires. We offer the framework as an open tool for mapping and analyzing distribution grids solely based on publicly-accessible data to support the construction and maintenance of reliable and clean energy systems around the world.

Beyond here and now: Evaluating pollution estimation across space and time from street view images with deep learning

Advances in computer vision, driven by deep learning, allows for the inference of environmental pollution and its potential sources from images. The spatial and temporal generalisability of image-based pollution models is crucial in their real-world application, but is currently understudied, particularly in low-income countries where infrastructure for measuring the complex patterns of pollution is limited and modelling may therefore provide the most utility. We employed convolutional neural networks (CNNs) for two complementary classification models, in both an end-to-end approach and as an interpretable feature extractor (object detection), to estimate spatially and temporally resolved fine particulate matter (PM2.5) and noise levels in Accra, Ghana. Data used for training the models were from a unique dataset of over 1.6 million images collected over 15 months at 145 representative locations across the city, paired with air and noise measurements. Both end-to-end CNN and object-based approaches surpassed null model benchmarks for predicting PM2.5 and noise at single locations, but performance deteriorated when applied to other locations. Model accuracy diminished when tested on images from locations unseen during training, but improved by sampling a greater number of locations during model training, even if the total quantity of data was reduced. The end-to-end models used characteristics of images associated with atmospheric visibility for predicting PM2.5, and specific objects such as vehicles and people for noise. The results demonstrate the potential and challenges of image-based, spatiotemporal air pollution and noise estimation, and that robust, environmental modelling with images requires integration with traditional sensor networks.

Implementing Deep Learning algorithms for urban tree detection and geolocation with high-resolution aerial, satellite, and ground-level images

Urban forests are becoming increasingly important for human well-being as they provide ecosystem services that contribute to improving well-being of city dwellers and to addressing climate change. However, despite their importance, there is an information gap in most of the world's urban forests due to the high cost and complexity of conducting standard forest inventories in urban environments. New technologies based on artificial intelligence can represent a smart and efficient alternative to costly traditional inventories. In this paper, we present an approach based on deep learning algorithms for the detection, counting, and geopositioning of trees using a combination of ground-level and aerial/satellite imagery. We tested several convolutional networks, exploring different combinations of hyperparameters and adjusting the query distance between ground-level images, detection radius, and various resolutions of satellite and aerial images. Our methodology is able to detect and accurately locate 79% of the urban street tree with a positional accuracy of 60 cm to the center of the canopy. Additionally, this approach allows us to determine the availability of photographs of urban trees, indicating from which Google Street View image each tree is visible. Our research provides a scalable and replicable solution to the scarcity of urban tree data and information worldwide, demonstrating the potential of artificial intelligence to revolutionize the way in which we inventory and monitor urban forests.

Artificial Intelligence based Building Attributes Enrichment in OpenStreetMap using Street-view Images

Artificial Intelligence based Building Attributes Enrichment in OpenStreetMap using Street-view Images