Validity and reliability of the virtual audit tool for estimating built-environment characteristics in Taiwan

Audit tools are useful for exploring the urban environment and its association with physical activity. Virtual auditing options are becoming increasingly available potentially reducing the resources needed to conduct these assessments. Only a few studies have explored the use of virtual audit tools. Our objective is to test if the Madrid Systematic Pedestrian and Cycling Environment Scan (M-SPACES) discriminates between areas with different urban forms and to validate virtual street auditing using M-SPACES. Three areas (N = 500 street segments) were selected for variation in population density. M-SPACES was used to audit street segments physically and virtually (Google Street View) by two researchers in 2013-2014. For both physical and virtual audits, all analyzed features score significantly different by area (p < 0.05). Most of the features showed substantial (ICC = 0.6-0.8) or almost perfect (ICC ≥ 0.8) agreement between virtual and physical audits, especially neighborhood permeability walking infrastructure, traffic safety, streetscape aesthetics, and destinations. Intra-rater agreement was generally acceptable (ICC > 0.6). Inter-rater agreement was generally poor (ICC < 0.4). Virtual auditing provides a valid and feasible way of measuring residential urban environments. Comprehensive auditor training may be needed to guarantee good inter-rater agreement.

Comparison of field and online observations for measuring land uses using the Microscale Audit of Pedestrian Streetscapes (MAPS)

BackgroundAltering micro-scale features of neighborhood walkability (e.g., benches, sidewalks, and cues of social disorganization or crime) could be a relatively cost-effective method of creating environments that are conducive to active living. Traditionally, measuring the micro-scale environment has required researchers to perform observational audits. Technological advances have led to the development of virtual audits as alternatives to observational field audits with the enviable properties of cost-efficiency from elimination of travel time and increased safety for auditors. This study examined the reliability of the Virtual Systematic Tool for Evaluating Pedestrian Streetscapes (Virtual-STEPS), a Google Street View-based auditing tool specifically designed to remotely assess micro-scale characteristics of the built environment.MethodsWe created Virtual-STEPS, a tool with 40 items categorized into 6 domains (pedestrian infrastructure, traffic calming and streets, building characteristics, bicycling infrastructure, transit, and aesthetics). Items were selected based on their past abilities to predict active living and on their feasibility for a virtual auditing tool. Two raters performed virtual and field audits of street segments in Montreal neighborhoods stratified by the Walkscore that was used to determine the ‘walking-friendliness’ of a neighborhood. The reliability between virtual and field audits (n = 40), as well as inter-rater reliability (n = 60) were assessed using percent agreement, Cohen’s Kappa statistic, and the Intra-class Correlation Coefficient.ResultsVirtual audits and field audits (excluding travel time) took similar amounts of time to perform (9.8 versus 8.2 min). Percentage agreement between virtual and field audits, and for inter-rater agreement was 80% or more for the majority of items included in the Virtual-STEPS tool. There was high reliability between virtual and field audits with Kappa and ICC statistics indicating that 20 out of 40 (50.0%) items had almost perfect agreement and 13 (32.5%) items had substantial agreement. Inter-rater reliability was also high with 17 items (42.5%) with almost perfect agreement and 11 (27.5%) items with substantial agreement.ConclusionsVirtual-STEPS is a reliable tool. Tools that measure the micro-scale environment are important because changing this environment could be a relatively cost-effective method of creating environments that are conducive to active living.

Virtual audits using Google Street View are an increasingly popular method of assessing neighborhood environments for health and urban planning research. However, the validity of these studies may be threatened by issues of image availability, image age, and variance of image age, particularly in the Global South. This study identifies patterns of Street View image availability, image age, and image age variance across cities in Latin America and assesses relationships between these measures and measures of resident socioeconomic conditions. Image availability was assessed at 530,308 near-road points within the boundaries of 371 Latin American cities described by the SALURBAL (Salud Urbana en America Latina) project. At the subcity level, mixed-effect linear and logistic models were used to assess relationships between measures of socioeconomic conditions and image availability, average image age, and the standard deviation of image age. Street View imagery was available at 239,394 points (45.1%) of the total sampled, and rates of image availability varied widely between cities and countries. Subcity units with higher scores on measures of socioeconomic conditions had higher rates of image availability (OR = 1.11 per point increase of combined index, p < 0.001) and the imagery was newer on average (− 1.15 months per point increase of combined index, p < 0.001), but image capture date within these areas varied more (0.59-month increase in standard deviation of image age per point increase of combined index, p < 0.001). All three assessed threats to the validity of Street View virtual audit studies spatially covary with measures of socioeconomic conditions in Latin American cities. Researchers should be attentive to these issues when using Street View imagery.

Neighborhood physical disorder is linked to deleterious resident physical and mental health. It is thus critical to develop low-cost, reliable methods that utilize publicly available imagery (e.g., Google Street View; GSV) to comprehensively audit neighborhoods. We aimed to create a reliable, efficient, and scale-flexible virtual audit of neighborhood disorder (Lot Assessment of Neighborhood Disorder; LAND) that can be aggregated to larger geographical units. A total of 710 block faces on 355 street segments were coded in Detroit, MI. We tested reliability between coders on 20% of the sample (71 segments (i.e., two sides of the street); 146 block faces (i.e., one side of the street) and found reliability was adequate at the individual lot (Kappas ranged from .60 - 1), block face (ICCs ranged from .94 -.98), and segment (ICCs ranged from .96-.98) levels, with the sole exception of graffiti (for which ICCs were typically in the .56 to .57 range). Moreover, LAND's score was positively correlated with number of vacant lots, area deprivation, and resident perceptions of their neighborhood. Overall, LAND evidence higher levels of reliability than previous physical neighborhood disorder virtual audits and evidenced significant correlations across several known correlates of physical neighborhood disorder, thus highlighting LAND as an effective virtual audit tool in the study of physical neighborhood disorder.

Recently, there has been a growing interest in developing new tools to measure neighborhood features using the benefits of emerging technologies. This study aimed to assess the psychometric properties of a neighborhood disorder observational scale using Google Street View (GSV). Two groups of raters conducted virtual audits of neighborhood disorder on all census block groups (N = 92) in a district of the city of Valencia (Spain). Four different analyses were conducted to validate the instrument. First, inter-rater reliability was assessed through intraclass correlation coefficients, indicating moderated levels of agreement among raters. Second, confirmatory factor analyses were performed to test the latent structure of the scale. A bifactor solution was proposed, comprising a general factor (general neighborhood disorder) and two specific factors (physical disorder and physical decay). Third, the virtual audit scores were assessed with the physical audit scores, showing a positive relationship between both audit methods. In addition, correlations between the factor scores and socioeconomic and criminality indicators were assessed. Finally, we analyzed the spatial autocorrelation of the scale factors, and two fully Bayesian spatial regression models were run to study the influence of these factors on drug-related police interventions and interventions with young offenders. All these indicators showed an association with the general neighborhood disorder. Taking together, results suggest that the GSV-based neighborhood disorder scale is a reliable, concise, and valid instrument to assess neighborhood disorder using new technologies.

BackgroundPhysical inactivity and unhealthy diet are key behavioral determinants underlying obesity. The neighborhood environment represents an important arena for modifying these behaviors, and hence reliable and valid tools to measure it are needed. Most existing virtual audit tools have been designed to assess either food or activity environments deemed relevant for adults. Thus, there is a need for a tool that combines the assessment of food and activity environments, and which focuses on aspects of the environment relevant for youth.ObjectiveThe aims of the present study were: (a) to adapt the SPOTLIGHT Virtual Audit Tool (S-VAT) developed to assess characteristics of the built environment deemed relevant for adults for use in an adolescent population, (b) to assess the tool’s inter- and intra-rater reliability, and (c) to assess its criterion validity by comparing the virtual audit to a field audit.MethodsThe tool adaptation was based on literature review and on results of a qualitative survey investigating how adolescents perceived the influence of the environment on dietary and physical activity behaviors. Sixty streets (148 street segments) in six neighborhoods were randomly selected as the study sample. Two raters assessed the inter- and intra-rater reliability and criterion validity, comparing the virtual audit tool to a field audit. The results were presented as percentage agreement and Cohen’s kappa (κ).ResultsIntra-rater agreement was found to be moderate to almost perfect (κ = 0.44–0.96) in all categories, except in the category aesthetics (κ = 0.40). Inter-rater agreement between auditors ranged from fair to substantial for all categories (κ = 0.24–0.80). Criterion validity was found to be moderate to almost perfect (κ = 0.56–0.82) for most categories, except aesthetics and grocery stores (κ = 0.26–0.35).ConclusionThe adapted version of the S-VAT can be used to provide reliable and valid data on built environment characteristics deemed relevant for physical activity and dietary behavior among adolescents.

Development and validation of automated microscale walkability audit method

Using Google Street View to Audit Neighborhood Environments

Virtual audits of streetscapes by crowdworkers.

There is increasing recognition that the neighborhood-built environment influences health outcomes, such as physical activity behaviors, and technological advancements now provide opportunities to examine the neighborhood streetscape remotely. Accordingly, the aims of this methodological study are to: (1) compare the efficiencies of physically and virtually conducting a streetscape audit within the neighborhood context, and (2) assess the level of agreement between the physical (criterion) and virtual (test) audits. Built environment attributes associated with walking and cycling were audited using the New Zealand Systematic Pedestrian and Cycling Environment Scan (NZ-SPACES) in 48 street segments drawn from four neighborhoods in Auckland, New Zealand. Audits were conducted physically (on-site) and remotely (using Google Street View) in January and February 2010. Time taken to complete the audits, travel mileage, and Internet bandwidth used were also measured. It was quicker to conduct the virtual audits when compared with the physical audits (χ = 115.3 min (virtual), χ = 148.5 min (physical)). In the majority of cases, the physical and virtual audits were within the acceptable levels of agreement (ICC ≥ 0.70) for the variables being assessed. The methodological implication of this study is that Google Street View is a potentially valuable data source for measuring the contextual features of neighborhood streets that likely impact on health outcomes. Overall, Google Street View provided a resource-efficient and reliable alternative to physically auditing the attributes of neighborhood streetscapes associated with walking and cycling. Supplementary data derived from other sources (e.g., Geographical Information Systems) could be used to assess the less reliable streetscape variables.

BackgroundThe opioid epidemic has ravaged rural communities in the United States. Despite extensive literature relating the physical environment to substance use in urban areas, little is known about the role of physical environment on the opioid epidemic in rural areas.ObjectiveThis study aimed to examine the reliability of Google Earth to collect data on the physical environment related to substance use in rural areas.MethodsSystematic virtual audits were performed in 5 rural Kentucky counties using Google Earth between 2017 and 2018 to capture land use, health care facilities, entertainment venues, and businesses. In-person audits were performed for a subset of the census blocks.ResultsWe captured 533 features, most of which were images taken before 2015 (71.8%, 383/533). Reliability between the virtual audits and the gold standard was high for health care facilities (>83%), entertainment venues (>95%), and businesses (>61%) but was poor for land use features (>18%). Reliability between the virtual audit and in-person audit was high for health care facilities (83%) and entertainment venues (62%) but was poor for land use (0%) and businesses (12.5%).ConclusionsPoor reliability for land use features may reflect difficulty characterizing features that require judgment or natural changes in the environment that are not reflective of the Google Earth imagery because it was captured several years before the audit was performed. Virtual Google Earth audits were an efficient way to collect rich neighborhood data that are generally not available from other sources. However, these audits should use caution when the images in the observation area are dated.

PurposeAuditors’ self-efficacy (SE) represents their level of confidence in improving their audit performance. This may be a crucial factor for auditors to perform effectively during the health crisis of COVID-19. This study aims to build on a social cognitive perspective to assess the SE of auditors during the coronavirus of 2019 (COVID-19) and to anticipate that mastery experience, verbal and social persuasion, vicarious experience, physiological and emotional states and virtual audit are determinants of auditors’ SE during COVID-19.Design/methodology/approachBased on data from Saudi Arabia, this study surveyed all auditors using an online questionnaire and collected 193 useful responses. Thus, this study analyzed the data using primary statistical tests and a structural equation model of partial least squares.FindingsThis study observes that auditors feel confident in their ability to perform audit activities as well during COVID-19 as at other times. This study also documents that VEs, physiological and emotional states and virtual audits play significant roles in SE. In further analyses, this study observes that auditors who are affiliated with big4 audit firms moderate the positive association between virtual audit and SE. All these results are verified under several econometrical appraisals and held constant.Originality/valueThis study provides a number of theoretical and practical implications.

Google street view image availability in the Bronx and San Diego, 2007-2020: Understanding potential biases in virtual audits of urban built environments.

BackgroundAlthough previous research has highlighted the association between the built environment and individual health, methodological challenges in assessing the built environment remain. In particular, many researchers have demonstrated the high inter-rater reliability of assessing large or objective built environment features and the low inter-rater reliability of assessing small or subjective built environment features using Google Street View. New methods for auditing the built environment must be evaluated to understand if there are alternative tools through which researchers can assess all types of built environment features with high agreement. This paper investigates measures of inter-rater reliability of GigaPan®, a tool that assists with capturing high-definition panoramic images, relative to Google Street View.MethodsStreet segments (n = 614) in Pittsburgh, Pennsylvania in the United States were randomly selected to audit using GigaPan® and Google Street View. Each audit assessed features related to land use, traffic and safety, and public amenities. Inter-rater reliability statistics, including percent agreement, Cohen’s kappa, and the prevalence-adjusted bias-adjusted kappa (PABAK) were calculated for 106 street segments that were coded by two, different, human auditors.ResultsMost large-scale, objective features (e.g. bus stop presence or stop sign presence) demonstrated at least substantial inter-rater reliability for both methods, but significant differences emerged across finely detailed features (e.g. trash) and features at segment endpoints (e.g. sidewalk continuity). After adjusting for the effects of bias and prevalence, the inter-rater reliability estimates were consistently higher for almost all built environment features across GigaPan® and Google Street View.ConclusionGigaPan® is a reliable, alternative audit tool to Google Street View for studying the built environment. GigaPan® may be particularly well-suited for built environment projects with study settings in areas where Google Street View imagery is nonexistent or updated infrequently. The potential for enhanced, detailed imagery using GigaPan® will be most beneficial in studies in which current, time sensitive data are needed or microscale built environment features would be challenging to see in Google Street View. Furthermore, to better understand the effects of prevalence and bias in future reliability studies, researchers should consider using PABAK to supplement or expand upon Cohen’s kappa findings.