Deep learning-based UAV image segmentation and inpainting for generating vehicle-free orthomosaic

Abstract

• A novel deep learning-based image segmentation and inpainting method was proposed to generate vehicle-free road orthomosaic. • This method is removing the unwanted objects from each original images captured by UAV before processing the orthomosaic. • By comparing the proposed and traditional ways, this study validated the performance of the UAV image inpainting to generate the vehicle-free orthomosaic. • The proposed method can remove provide contextual information more effectively by eliminating the unwanted objects. A low-altitude orthomosaic derived by an unmanned aerial vehicle (UAV) has been widely utilized for various purposes in large-scale infrastructure management. However, unwanted objects, such as cars and trucks, captured in the aerial images captured by the UAV have negative impacts on the quality of orthomosaic. To this end, this study presented a novel method to remove the effect of unwanted objects on UAV-generated orthomosaic. The proposed method applied a deep learning-based image segmentation and inpainting algorithm to remove the vehicles from individual UAV images before processing structure from motion (SfM), and then it resulted in generateing an orthomosaic with the inpainted UAV images. To validate the proposed method, this study conducted a case study in actual highway environment and compared the performance of the proposed method with that of another method, which directly removes and inpaints vehicles from the final orthomosaic. Through comparison tests, it is shown that the proposed method is more effective than the other. The proposed automatic vehicle-free orthomosaic generation method can contribute to creating up-to-date immersive content for transportation infrastructure management.