Large-Scale Synthetic Urban Dataset for Aerial Scene Understanding

Qian Gao,Wensheng Niu,Xukun Shen

doi:10.1109/access.2020.2976686

Abstract

The geometric extraction and semantic understanding in bird's eye view plays an important role in cyber-physical-social systems (CPSS), because it can help human or intelligent agents (IAs) to perceive larger range of environment. Moreover, due to lack of comprehensive dataset from oblique perspective, fog-end deep learning algorithms for this purpose is still in blank. In this paper, we propose a novel method to generate synthetic large-scale dataset for geometric and semantic urban scene understanding from bird's eye view. There are two main steps involved, one is modeling and the other is rendering, which are processed by CityEngine and UnrealEngine4 respectively. In this way, synthetic aligned multi-model data are obtained efficiently, including spectral images, semantic labels, depth and normal maps. Specifically, terrain elevation, street graph, building style and trees distribution are all randomly generated according realistic situation, a few of handcrafted semantic labels annotated by colors spread throughout the scene, virtual cameras moved according to realistic trajectories of unmanned aerial vehicles (UAVs). For evaluation of practicability of our dataset, we manually labeled tens of aerial images downloaded from internet. And the experiment result show that, in both pure and combined mode, the dataset can improve the performance significantly.

Highlights

With the rapid development of the internet of things (IoT) technology, cyber-physical-socal Systems (CPSS) have gradually taken shape
Inspired by SYNTHIA [5], we proposes a comprehensive large-scale urban scene synthesis dataset
In response to the above problems and requirements, this paper proposes a method for generating large-scale synthetic aerial datasets based on programmatic modeling and photorealistic rendering

Summary

INTRODUCTION

With the rapid development of the internet of things (IoT) technology, cyber-physical-socal Systems (CPSS) have gradually taken shape. The advantage of synthetic data sets is batch production In this problem, we need to pay attention to the settings of the virtual camera, and only realistically simulate the UAVs, including the angle, field of view, and height. This is one of the main advantages of synthetic datasets, which can accurately register multiple data models together For this problem, depth maps, semantic labels, and light maps can all be used for 2D/3D scene understanding. In response to the above problems and requirements, this paper proposes a method for generating large-scale synthetic aerial datasets based on programmatic modeling and photorealistic rendering. This method has the characteristics of high degree of automation, rapid generation, realistic image and multi-modal data.

RELATED WORKS

EXPERIMENTS

Findings

CONCLUSION AND FUTURE WORKS