Restoration of ancient temple murals using cGAN and PConv networks

Abstract

India is a fountainhead of several art forms like paintings, inscriptions, sculptures, pottery, textile arts, and so on. Mural paintings are one in all of them and are usually seen on the walls of temples and caves. A redeeming feature of temple murals is that the architectural elements of the given space are simultaneously incorporated into the paintings. The majority of these paintings have faded with time, and a select handful have cracks and dirt stains on top of them. Understanding the nuances of the paintings become difficult as they deteriorate further. These paintings need to be restored by qualified craftsmen, who are difficult to find today. Therefore, a powerful image restoration method is needed to meet the requirements of the mural paintings. Thus an efficient inpainting technique for the reconstruction of ancient temple murals that ignores these multiple random irregularities is being proposed. The proposed method makes use of cGAN for both the automatic development of masks and the identification of degraded sections. A mask is an example of a black and white image where the white pixels require some editing while the black pixels do not. These masks are used by the algorithm as a hyper-parameter to forecast which patch has to be filled in next. The deteriorated murals are rebuilt using a sliding window-based Deep Convolutional Network, where the convolution is masked and renormalized to be conditioned on only valid pixels. As part of the forward pass, the proposed work automatically creates a changed mask for the following layer. The performance of our combined cGAN-Deepconv inpainting technique has been compared with six state-of-the-art inpainting methods. The experimental reconstruction results confirmed that our Sliding-window-Deepconv inpainting is more adaptable and better suited for mural restoration. Further, the proposed method achieved the best reconstruction results with the absolute best values of several performance parameters, i.e. Peak Signal to Noise Ratio, Mean Squared Error, and Structural Similarity Index.