Object Detection-Based System for Traffic Signs on Drone-Captured Images

Abstract

The construction industry is on the path to digital transformation. One of the main challenges in this process is inspecting, assessing, and maintaining civil infrastructures and construction elements. However, Artificial Intelligence (AI) and Unmanned Aerial Vehicles (UAVs) can support the tedious and time-consuming work inspection processes. This article presents an innovative object detection-based system which enables the detection and geo-referencing of different traffic signs from RGB images captured by a drone’s onboard camera, thus improving the realization of road element inventories in civil infrastructures. The computer vision component follows the typical methodology for a deep-learning-based SW: dataset creation, election and training of the most accurate object detection model, and testing. The result is the creation of a new dataset with a wider variety of traffic signs and an object detection-based system using Faster R-CNN to enable the detection and geo-location of traffic signs from drone-captured images. Despite some significant challenges, such as the lack of drone-captured images with labeled traffic signs and the imbalance in the number of images for traffic signal detection, the computer vision component allows for the accurate detection of traffic signs from UAV images.