Street View Imagery Research Articles

Exploring the spatial association between the distribution of temperature and urban morphology with green view index.

Urban heat islands will occur if city neighborhoods contain insufficient green spaces to create a comfortable environment, and residents' health will be adversely affected. Current satellite imagery can only effectively identify large-scale green spaces and cannot capture street trees or potted plants within three-dimensional building spaces. In this study, we used a deep convolutional neural network semantic segmentation model on Google Street View to extract environmental features at the neighborhood level in Taipei City, Taiwan, including the green vegetation index (GVI), building view factor, and sky view factor. Monthly temperature data from 2018 to 2021 with a 0.01° spatial resolution were used. We applied a linear mixed-effects model and geographically weighted regression to explore the association between pedestrian-level green spaces and ambient temperature, controlling for seasons, land use information, and traffic volume. Their results indicated that a higher GVI was significantly associated with lower ambient temperatures and temperature differences. Locations with higher traffic flows or specific land uses, such as religious or governmental, are associated with higher ambient temperatures. In conclusion, the GVI from street-view imagery at the community level can improve the understanding of urban green spaces and evaluate their effects in association with other social and environmental indicators.

GreenR: An open-source framework for quantifying urban greenness

How do we quantify the levels of greenness within urban street networks? Numerous attempts to quantify this factor have been made through survey methodologies, remote sensing data, and street view imagery. The results are promising, but are often limited by scalability constraints, including limited data availability, high requirements of computational power, and validation challenges. This study introduces a comprehensive framework for urban greenness assessment, leveraging open-source data to overcome these limitations. Central to this framework is the development of the greenR R package and an accompanying Shiny app, designed to compute, visualize, and analyze a novel proximity-based green index for individual street segments. Beyond this, the framework innovatively extends its analytical capability by integrating accessibility analysis, Green View Index quantification as well as introducing the Green Space Similarity Index, a novel metric that evaluates and compares the characteristics of urban green spaces across different regions. This extension enriches the proposed framework, providing not just a measure of proximity to green spaces, but also insights into their spatial connectivity and distribution. The efficacy of greenR is demonstrated by studying urban greenness patterns across several cities, highlighting the potential impact of such an open-source framework for citizens, urban planners, and policy-makers. This study not only advances our methodological approach to quantifying urban greenness but also provides practical tools and metrics that can inform sustainable urban planning and policy decisions.

Day-to-Night Street View Image Generation for 24-Hour Urban Scene Auditing Using Generative AI.

A smarter city should be a safer city. Nighttime safety in metropolitan areas has long been a global concern, particularly for large cities with diverse demographics and intricate urban forms, whose citizens are often threatened by higher street-level crime rates. However, due to the lack of night-time urban appearance data, prior studies based on street view imagery (SVI) rarely addressed the perceived night-time safety issue, which can generate important implications for crime prevention. This study hypothesizes that night-time SVI can be effectively generated from widely existing daytime SVIs using generative AI (GenAI). To test the hypothesis, this study first collects pairwise day-and-night SVIs across four cities diverged in urban landscapes to construct a comprehensive day-and-night SVI dataset. It then trains and validates a day-to-night (D2N) model with fine-tuned brightness adjustment, effectively transforming daytime SVIs to nighttime ones for distinct urban forms tailored for urban scene perception studies. Our findings indicate that: (1) the performance of D2N transformation varies significantly by urban-scape variations related to urban density; (2) the proportion of building and sky views are important determinants of transformation accuracy; (3) within prevailed models, CycleGAN maintains the consistency of D2N scene conversion, but requires abundant data. Pix2Pix achieves considerable accuracy when pairwise day-and-night-night SVIs are available and are sensitive to data quality. StableDiffusion yields high-quality images with expensive training costs. Therefore, CycleGAN is most effective in balancing the accuracy, data requirement, and cost. This study contributes to urban scene studies by constructing a first-of-its-kind D2N dataset consisting of pairwise day-and-night SVIs across various urban forms. The D2N generator will provide a cornerstone for future urban studies that heavily utilize SVIs to audit urban environments.

Exploring non-linear effects of environmental factors on the volume of pedestrians of different ages using street view images and computer vision technology

Creating pedestrian-friendly neighborhoods and encouraging walking activities not only improves urban liveliness but also delivers health and environmental benefits. Previous research has largely focused on individual walking behavior, which often exhibits stronger associations with personal traits than with built environment characteristics. Pedestrian volume, a significant indicator of urban vitality and collective walking behavior, may have a stronger relationship with environmental characteristics. Moreover, prior studies often hypothesize a linear relationship between built environmental determinants and walking behavior. The exploration of potential non-linear influences may help policy makers and urban planners by identifying minimum, maximum, and optimal values of built environmental variables conducive for walking. Furthermore, studies focusing on the collective walking of older pedestrians remain scarce in the context of aging societies.In this study, we utilize street view imagery and advanced computer vision algorithms to estimate citywide pedestrian volumes and their corresponding age classifications (older adults vs. all). Additionally, we assessed both micro and macro built environmental factors. The possible non-linear impact is examined using Gradient Boosting Decision Tree (GBDT). Our findings reveal disparities in the influence of environmental determinants on the volume of older pedestrians versus that of all pedestrians. Also, the significance of environmental elements exhibits variations across different spatial resolutions. Further, both eye-level vegetation and building level show an inverted U-shaped influence on the pedestrian count.

Fusion of satellite and street view data for urban traffic accident hotspot identification

As the number of vehicles and the volume of traffic swell in urban centers, cities have experienced a concomitant increase in traffic accidents. Proactively identifying accident-prone hotspots in urban environments holds the promise of preventing traffic mishaps, thereby curtailing the incidence of accidents and reducing property damage. This research introduces the Two-Branch Contextual Feature-Guided Converged Network (TCFGC-Net) utilizing multimodal satellite and street view data. Designed to extract global structural features from satellite imagery and dynamic continuous features from street view imagery, the model aims to improve the accuracy of detecting urban accident hotspots. For the satellite imagery branch, we propose the Contextual Feature Coupled Convolutional Neural Network (Trans-CFCCNN) designed to extract global spatial features and discern feature correlations across adjacent regions. For the street view imagery branch, we develop the Sequential Feature Recurrent Attention Network (SFRAN) to assimilate and integrate dynamic scene features captured from successive street view images. We designed the Multi-Branch Feature Adaptive Fusion Structure (MBFAF) to aggregate different branch features for accurate identification of accident hotspots. Experimental results show that the model performs well, with an overall accuracy of 93.7 %. Ablation studies confirm that relative to standalone street view and satellite branch analyses, implementing multimodal fusion enhances the model's accuracy by 12.05 % and 17.86 %, respectively. The innovative fusion structure proposed herein garners a 4.22 % increase in model accuracy, outpacing conventional feature concatenation techniques. Furthermore, the model outperforms existing deep learning models in terms of overall efficacy. Additionally, to showcase the efficacy of the proposed model structure, we utilize Class Activation Maps (CAM) to provide visual interpretability for the model. These results suggest that the dual-branch fusion model effectively decreases false alarm occurrences and directs the model's focus toward regions more pertinent to accident hotspots. Finally, the code and model used for identifying hotspots of urban traffic accidents in this study are available for access: https://github.com/gwt-ZJU/TCFGC-Net.

Integrating street view imagery and taxi trajectory for identifying urban function of street space

ABSTRACT Street space is a crucial component of public space, serving as a site for a variety of human activities. However, prior studies have primarily focused on the traffic function of street space, neglecting other functional types, such as residential and commercial. To address this gap, this study proposes a classification method for street space by integrating taxi trajectory and street view imagery. The dynamic travel features of the residents are extracted from taxi trajectory data and the multi-level physical environment features of the streets are constructed based on street view imagery data. Then, the street space with same urban functions is assigned to the same cluster based on the dynamic travel features of the residents, the multi-level physical environment features of streets and K-means method. The proposed method is empirically applied to the Futian District of Shenzhen to demonstrate its validity, and the street space is successfully classified into five types, namely public, traffic, commercial, residential, and mixed commercial and residential. This work contributes to the evaluation of street space quality, facilitating scientific planning and efficient governance of street space.

Combining Deep Learning and Street View Imagery to Map Smallholder Crop Types

Accurate crop type maps are an essential source of information for monitoring yield progress at scale, projecting global crop production, and planning effective policies. To date, however, crop type maps remain challenging to create in low- and middle-income countries due to a lack of ground truth labels for training machine learning models. Field surveys are the gold standard in terms of accuracy but require an often-prohibitively large amount of time, money, and statistical capacity. In recent years, street-level imagery, such as Google Street View, KartaView, and Mapillary, has become available around the world. Such imagery contains rich information about crop types grown at particular locations and times. In this work, we develop an automated system to generate crop type ground references using deep learning and Google Street View imagery. The method efficiently curates a set of street-view images containing crop fields, trains a model to predict crop types using either weakly-labeled images from disparate out-of-domain sources or zero-shot labeled street view images with GPT-4V, and combines the predicted labels with remote sensing time series to create a wall-to-wall crop type map. We show that, in Thailand, the resulting country-wide map of rice, cassava, maize, and sugarcane achieves an accuracy of 93%. We publicly release the first-ever crop type map for all of Thailand 2022 at 10m-resolution with no gaps. To our knowledge, this is the first time a 10m-resolution, multi-crop map has been created for any smallholder country. As the availability of roadside imagery expands, our pipeline provides a way to map crop types at scale around the globe, especially in underserved smallholder regions.

CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series

Urban transformations have profound societal impact on both individuals and communities at large. Accurately assessing these shifts is essential for understanding their underlying causes and ensuring sustainable urban planning. Traditional measurements often encounter constraints in spatial and temporal granularity, failing to capture real-time physical changes. While street view imagery, capturing the heartbeat of urban spaces in a pedestrian point of view, can add as a high-definition, up-to-date, and on-the-ground visual proxy of urban change. We curate the largest street view time series dataset to date, and propose an end-to-end change detection model to effectively capture physical alterations in the built environment at scale. We demonstrate the effectiveness of our proposed method by benchmark comparisons with previous literature and implementing it at the city-wide level. Our approach has the potential to supplement existing dataset and serve as a fine-grained and accurate assessment of urban change.

Abstract MP76: Street View Greenspace Exposures and Cardiovascular Health Among Children - Evidence From Project Viva

Introduction: Research on greenspace and cardiovascular health (CVH) in children is limited, and most previous studies rely on coarse satellite-based measurements of greenspace, which do not capture exposure as individuals experience it. Street View imagery (SVI) enables measurement of greenspace from a ground-level perspective and captures specific elements of greenspace that may be most relevant to health. This study aimed to examine associations of SVI-based greenspace with CVH in children in Project Viva, an eastern Massachusetts-based cohort. Methods: Using data from 1,163 children, we applied deep learning segmentation algorithms to SVI from 2007-2010 to derive metrics of greenspace (%of trees, or grass) within a 500m radius of participants’ residential addresses obtained during mid-childhood (mean age 7.9y). At mid-childhood and early adolescence (mean age 13.2y), we calculated CVH scores (range 0-100 units; higher scores represent better CVH) using the American Heart Association’s Life’s Essential 8 algorithm, based on 4 biomedical measures (body mass index, blood lipids, blood glucose, and blood pressure) and 4 behavioral measures (heart healthy levels of diet, physical activity, avoidance of nicotine, and sleep). Using linear regression, we examined associations of mid-childhood SVI-based greenspace metrics with mid-childhood (cross-sectional) and early adolescence (longitudinal) CVH scores, adjusting for child’s sex, race and ethnicity, and age at outcome, as well as an index score of socioeconomic status comprising individual and neighborhood metrics. Results: Among the 1163 participants, 65% were non-Hispanic White and 49% were female. The mean (SD) LE8 score was 83.3 (8.0) in mid-childhood and 81.7 (8.9) in early adolescence. In the unadjusted models, a higher %trees and grass per SD, as measured by SVI within children’s residence in mid-childhood, was associated with a higher overall CVH score (β 1.63 [95%CI: 0.93, 2.33] for trees; β 1.10 [95%CI: 0.36, 1.83] for grass). In longitudinal analyses, the results were similar (β 1.69 [95%CI: 0.87, 2.52] for trees; β 0.82 [95%CI: -0.01, 1.64] for grass). In fully-adjusted analyses, the cross-sectional and longitudinal results for %trees and grass were attenuated and not statistically significant. However, when CVH scores were disaggregated by behavioral and biomedical sub-scores, %trees was positively associated with behavioral sub-scores in cross-sectional fully-adjusted analyses at mid-childhood (β 0.89 [95%CI: 0.32, 1.46]). Conclusion: Among participants in Project Viva, SVI-based metrics of greenspace exposure in mid-childhood were cross-sectionally associated with behavioral factors promoting higher CVH. Greenspace may hold promise for improving children’s CVH through the behavioral pathway and should be considered in future health-promotion efforts.

Evaluating implied urban nature vitality in San Francisco: An interdisciplinary approach combining census data, street view images, and social media analysis

Urban green spaces (UGS) are vital in modern cities, offering extensive health, social, and environmental benefits. However, traditional research methods primarily focus on UGS distribution and aggregation through 2D mapping, often neglecting the quality and vitality of urban natural environments. This limited approach hampers our full understanding of the complex issues and opportunities surrounding UGS. This study proposes a novel concept of Implied Urban Nature Vitality (IUNV) and evaluation framework that offers a comprehensive lens to understand better and evaluate the manifold human-urban-nature interactions in modern cityscapes. Based on our IUNV framework, an interdisciplinary investigation is conducted to show the distribution and population-level perceived IUNV in San Francisco by leveraging a triad of data sources: census, street-built environment, and social media data. Utilizing census data, we analyze socio-economic influences on UGS distribution and IUNV, including factors such as education, age demographics, income, and ethnicity. Street view imagery (SVI), analyzed with advanced image recognition algorithms, serves as a proxy for visual and physical aspects of IUNV, highlighting features like trees, sky, buildings, and roads. This analysis paints a granular picture of UGS's spatial distribution and physical attributes, facilitating an objective measure of IUNV. Subsequently, we analyze Flickr photos related to urban natural areas, examining their distribution and identifying thematic clusters that illuminate various aspects of UGS vitality. Lastly, we combine computer vision and manual review to define 12 IUNV themes from architecture and nature, eco-friendly gatherings, to cultural performance, exploring the relationship between the vitality clusters and the independent variables. The main findings are: (1) Macro-level factors (e.g., accessibility level, land use mix level, road density, population density, etc.) are the dominant variables influencing IUNV.; (2) Street view factors play key roles in IUNV. Through this holistic IUNV analysis, the study shed light on the complexities of urban green space planning and management, informing future urban development strategies towards greater vitality and, by extension, environmental and social sustainability.

A representation-learning approach for insurance pricing with images

AbstractUnstructured data are a promising new source of information that insurance companies may use to understand their risk portfolio better and improve the customer experience. However, these novel data sources are difficult to incorporate into existing ratemaking frameworks due to the size and format of the unstructured data. This paper proposes a framework to use street view imagery within a generalized linear model. To do so, we use representation learning to extract an embedding vector containing useful information from the image. This embedding is dense and low dimensional, making it appropriate to use within existing ratemaking models. We find that there is useful information included in street view imagery to predict the frequency of claims for certain types of perils. This model can be used as in a ratemaking framework but also opens the door to future empirical research on attempting to extract which characteristics within the image leads to increased or decreased predicted claim frequencies. Throughout, we discuss the practical difficulties (technical and social) of using this type of data for insurance pricing.

Examining the relationship between active transport and exposure to streetscape diversity during travel: A study using GPS data and street view imagery

Active transport (AT)—physical activity (PA) during travel—can promote human health. Among built environment factors related to travel research, design refers to the street network features encouraging AT. The advent of street view images (SVIs) presents the potential to measure design during travel by capturing the eye-level built environments. Benefited by SVIs, this study innovatively introduces streetscape diversity—the interconnection of street view-derived built environment factors—during travel as the proxy to measure design from the street view perspective. Exposures to higher streetscape diversity could provide increased access to potential destinations and therapeutic landscapes, thereby promoting AT. Through integrating SVIs and young adults’ Global Positioning System (GPS) trajectories, this study utilized negative binomial regression models to assess the relationship between streetscape diversity and time spent in AT. Results indicate that the inclusion of streetscape diversity improves the model performance, and there is a positive relationship between streetscape diversity and AT. This finding indicates that increased access to travel routes with diverse streetscapes could increase the probability of participating in AT. Furthermore, integrating individual GPS data and SVIs allows more precise space-time measurement of individual environmental exposures.

Measuring cyclists’ subjective perceptions of the street riding environment using K-means SMOTE-RF model and street view imagery

Cyclists' willingness to ride is usually influenced by their subjective perception of the street riding environment. Measuring this perception is crucial for enhancing residents' willingness to ride. We propose an SSB framework (Public Security, Traffic Safety, Scenic Beauty) to quantify cyclists' subjective perception using street view imagery (SVI) and volunteer rating data. To address the issue of imbalanced class distribution in the volunteer rating data and enhance the model's ability to distinguish between positive and negative perception scenes, we employed a combination of the Kmeans Synthetic Minority Over-Sampling Technique (Kmeans-SMOTE) and the Random Forest (RF) classifier. The Kmeans SMOTE-RF model improved Area Under the Curve (AUC) by 0.327 for public safety, 0.2 for traffic safety, and 0.209 for scenic beauty compared to the RF model. Additionally, we incorporated Shapley Additive Explanations (SHAP) to examine how the visual features of SVI impact cyclists' subjective perception. Trees had a positive impact on all dimensions. Fence and sidewalk were key features for enhancing traffic safety perception, while roads positively affected public security and scenic beauty. These insights support urban planners in understanding the relationship between SVI features and cyclists' perceptions, aiding the design of cyclist-friendly street environments.

Online Street View-Based Approach for Sky View Factor Estimation: A Case Study of Nanjing, China

The Sky View Factor (SVF) stands as a critical metric for quantitatively assessing urban spatial morphology and its estimation method based on Street View Imagery (SVI) has gained significant attention in recent years. However, most existing Street View-based methods prove inefficient and constrained in SVI dataset collection. These approaches often fall short in capturing detailed visual areas of the sky, and do not meet the requirements for handling large areas. Therefore, an online method for the rapid estimation of a large area SVF using SVI is presented in this study. The approach has been integrated into a WebGIS tool called BMapSVF, which refines the extent of the visible sky and allows for instant estimation of the SVF at observation points. In this paper, an empirical case study is carried out in the street canyons of the Qinhuai District of Nanjing to illustrate the effectiveness of the method. To validate the accuracy of the refined SVF extraction method, we employ both the SVI method based on BMapSVF and the simulation method founded on 3D urban building models. The results demonstrate an acceptable level of refinement accuracy in the test area.

Housing Perceptions and Code Enforcement: An Assessment of Demolition Orders Using Street View Imagery and Machine Intelligence

The rapid growth of U.S. Sunbelt cities has intensified urban development pressures. Low-income housing demolitions are a result of such pressures as they are “low hanging fruit” for urban renewal, which can be further intensified by housing quality perceptions. By combining deep learning on Street View images (STV) with machine learning, we provide a model that accurately predicts demolition orders and allows assessing the heterogeneity of these predictions depending on the evaluator’s perceptions. Based on fast-growing San Antonio (TX) data, our results show that automated models can be useful to assess human perception biases of code enforcers.

Non-linear and synergistic effects of built environment factors on older adults’ walking behavior: An analysis integrating LightGBM and SHAP

Examining the relationship between the built environment and older adults’ walking behavior is of critical importance for the development of aging-friendly cities and communities. Previous studies, however, have paid limited attention to the non-linear and synergistic effects of built environment factors. To this end, based on multi-source data such as the Travel Characteristic Survey of Hong Kong and Google Street View imagery, this study integrates two advanced machine learning models—light gradient-boosting machine (LightGBM) and SHapley Additive exPlanations (SHAP)—to analyze the non-linear and synergistic effects of various built environment factors on older adults’ walking time. The results show that the effect of the built environment is largely non-linear. Critical built environment factors include access to recreational facilities and land-use mix. Access to metro and parks, however, plays a marginal role in affecting older adults’ walking. Furthermore, the synergistic effects of built environment variable pairs (e.g., access to recreational facilities and intersection density) are also identified.

How Mobility and Temporal Contexts May Affect Environmental Exposure Measurements: Using Outdoor Artificial Light at Night (ALAN) and Urban Green Space as Examples

Temporal contexts are essential for the derivation of causally relevant environmental exposure in environmental and public health studies. We argue that the proper temporal contexts need further emphasis in the mobility-oriented research paradigm and articulate this issue using a study in Hong Kong. We use people’s exposure to outdoor artificial light at night (ALAN) and green space as two essential examples. We recruited 208 participants from two representative communities in Hong Kong and derived their mobility-oriented environmental exposures from high-resolution remote sensing data and Google Street View imagery. We employed one-standard-deviational ellipses to quantitatively represent participants’ activity spaces, and we further used participants’ seven-day Global Positioning System trajectories to derive their spatiotemporally weighted exposures to green space and outdoor ALAN in different temporal contexts. Multiple t tests were used to examine the disparities in activity spaces and measured exposures in different temporal contexts, and these exposure measurements were then used to predict people’s health outcomes. We found that people’s activity spaces are significantly different in size between day and night, for both weekdays and weekends, and in both geographic contexts. We further observed that improper temporal contexts could lead to significantly different environmental exposure levels and causally irrelevant modeling results.

Associations between community green view index and fine particulate matter from Airboxes

Urban greenery can help to improve air quality, reduce health risks and create healthy livable urban communities. This study aimed to explore the role of urban greenery in reducing air pollution at the community level in Tainan City, Taiwan, using air quality sensors and street-view imagery. We also collected the number of road trees around each air quality sensor site and identified the species that were best at absorbing PM2.5. Three greenness metrics were used to assess community greenery in this study: two Normalized Difference Vegetation Indices (NDVI) from different satellites and the Green View Index (GVI) from Google Street View (GSV) images. Land-use Regression (LUR) was used for statistical analysis. The results showed that a higher GVI within a 500 m buffer was significantly associated with decreased PM2.5. Neither NDVI metrics within a 500 m circular buffer were significantly associated with decreased PM2.5. Evergreen trees were significantly associated with lower ambient PM2.5, compared with deciduous and semi-deciduous trees. Because localized changes in air quality profoundly affect public health and environmental equity, our findings provide evidence for future urban community greenspace planning and its beneficial impacts on reducing air pollution.

Street view imagery: AI-based analysis method and application

Street view imagery is an emerging form of geographic big data. It presents urban visual environments from the perspective of urban residents and also contains non-visual environment of cities, such as urban human activities and socio-economic development. However, traditional digital image processing has its limitations, and the continuous development of artificial intelligence, especially computer vision and deep learning, provides strong technical support for exploring the rich semantic information in street view imagery. This paper reviews the related research on street view imagery and its artificial intelligence analysis methods and applications. It outlines the acquisition, storage, and common data sources of street view imagery. Then it introduces computer vision, deep learning, and commonly used open-source datasets in street view imagery analysis. It also detailed three aspects of AI-based street view imagery applications, namely quantification of the physical space, urban perception, and spatial semantic speculation. Finally, issues like data acquisition, domain adaption and deep learning black box are discussed. The hotspots and prospects for the development of this research topic are also prospected.

Robust Building Identification from Street Views Using Deep Convolutional Neural Networks

Street view imagery (SVI) is a rich source of information for architectural and urban analysis using computer vision techniques, but its integration with other building-level data sources requires an additional step of visual building identification. This step is particularly challenging in architecturally homogeneous, dense residential streets featuring narrow buildings, due to a combination of SVI geolocation errors and occlusions that significantly increase the risk of confusing a building with its neighboring buildings. This paper introduces a robust deep learning-based method to identify buildings across multiple street views taken at different angles and times, using global optimization to correct the position and orientation of street view panoramas relative to their surrounding building footprints. Evaluating the method on a dataset of 2000 street views shows that its identification accuracy (88%) outperforms previous deep learning-based methods (79%), while methods solely relying on geometric parameters correctly show the intended building less than 50% of the time. These results indicate that previous identification methods lack robustness to panorama pose errors when buildings are narrow, densely packed, and subject to occlusions, while collecting multiple views per building can be leveraged to increase the robustness of visual identification by ensuring that building views are consistent.