Restoration of artwork using deep neural networks

Abstract

Paintings and other similar work of art represent an important part of our heritage and contemporary culture. However, due to the nature of materials used in these works, they are prone to damage and degradation over a period of time. Some of damages to these works may include torn canvases, smudges, exposure to elements etc. This necessitates the need for restoration of artworks. The restoration process is very time consuming and is a delicate task making it prone to human error. The virtual restoration of digitized artworks can be very helpful in this process. In this paper, we have proposed a method based on deep neural networks for virtual restoration of the digitized artworks. The paper presents a hybrid model which employs automatic mask generation based on Mask R-CNN and image inpainting using U-Net architecture with partial convolutions and automatic mask update. The proposed approach is evaluated qualitatively as well as quantitatively. The qualitative evaluation of the approach is done by engaging three domain art experts. On the other hand, quantitative validation of the proposed method is done using dataset of images having artificially created irregular holes by employing mean square error (MSE) and structural similarity index (SSIM) metrics. The results obtained show that the proposed approach is quite effective in virtual restoration of the digitized artworks.