Removal of occluding construction workers in job site image data using U-Net based context encoders

Abstract

In the field of Architecture, Engineering, Construction, and Facility Management (AEC/FM), job site surveying is critical for tasks such as construction progress monitoring and quality assurance and quality control. Moving obstacles often occlude the sight of view, and sometimes block the view of interest. However, images are often collected for documentation and analysis of the building under construction or operation. The occluding objects, such as the construction workers, compromise the collected image data. On the other hand, the capability of the Context Encoders model took the pioneering step providing the opportunity of image inpainting. This research applies Context Encoders to remove redundant objects in images and inpaints the background context. The original Context Encoders model, a deep learning model, requires a large training dataset for a decent image inpainting. In the model, the region in need of context inpainting is constrained to be a predefined area, fixed in size and position in the image. In this work, we adapted a deep learning architecture, U-Net, for a direct image-to-image translation for context inpainting, and thus relaxing the fixed size and position constraints in the original Context Encoders model. Simultaneously in a single framework, the proposed model not only erases redundant objects but also inpaints the missing content in the pixel scale, based on surrounding semantic clues. Employing the U-Net yields encouraging results for cases in operating buildings and those under construction.