Dual attention and channel transformer based generative adversarial network for restoration of the damaged artwork

Abstract

Artworks are treasures of valuable cultural and historical heritage. Artworks get damaged due to environmental and other factors. The artificial intelligence-based restoration of digitized artwork images can guide the artists in physically restoring the damaged artworks. Previous methods have not been able to restore artwork images well. This paper proposes a dual (spatial and channel) attention and channel transformer-based generative adversarial network to restore damaged artwork images digitally. The proposed generative adversarial network has spatial and channel attention layers in the encoder part of the generator and a channel transformer between skip connections from the encoder to the decoder part of the generator. Spatial and channel attention helps learn inter-spatial and inter-channel global relationships among image features. Channel transformer ensures multiscale feature fusion and reduces the semantic gap between encoder and decoder layer features. Moreover, the proposed network has been trained using a linear combination of perceptual, adversarial, and structured similarity index measure loss, which helps better train the network. Further, the proposed network has been validated on two different datasets, and the results indicate that the proposed method outperforms state-of-the-art artwork restoration methods.