Smart Curb Digital Twin: Inventorying Curb Environments Using Computer Vision and Street Imagery

Abstract

Digital Twin (DT) offers a novel framework to track, model, analyze, and anticipate complex urban processes and support data-driven decision-making. However, a premise of developing DT applications is to inventory physical urban built environment digitally, which are often lacking for small-and medium-sized cities due to limited resources. Particularly, few digital inventories have been built for urban curb environments, which have been increasingly challenged by new vehicle technologies and emerging mobility services. We propose a data-driven framework to inventory curb facilities across types and locations using computer vision (CV) and Google Street View (GSV) imagery. Specifically, we used a state-of-the-art semantic segmentation model, i.e., DeepLab V3, pre-trained on the CityScapes dataset, to detect curb facilities of interest from GSV images. We then used the Inverse Perspective Mapping (IPM) to estimate the spatial location for each detected facility and used spatial processing to aggregate and filter estimation results. We demonstrated the framework for inventorying curbs in the Innovation District in the City of Gainesville, FL. The preliminary research contributes to Smart Curb Digital Twin for more safe, accessible, and productive curb environments.