A Connected Auto-Encoders Based Approach for Image Separation with Side Information: With Applications to Art Investigation

Abstract

X-radiography is a widely used imaging technique in art investigation, whether to investigate the condition of a painting or provide insights into artists’ techniques and working methods. In this paper, we propose a new architecture based on the use of ‘connected’ auto-encoders in order to separate mixed X-ray images acquired from double-sided paintings, where in addition to the mixed X-ray image one can also exploit the two RGB images associated with the front and back of the painting. This proposed architecture uses convolutional auto-encoders that extract features from the RGB images that can be employed to (1) reproduce both of the original RGB images, (2) reconstruct the associated separated X-ray images, and (3) regenerate the mixed X-ray image. It operates in a totally self-supervised fashion without the need for examples containing both the mixed X-ray images and the separated ones. Based on images from the double-sided wing panels from the famous Ghent Altarpiece, painted in 1432 by the brothers Hubert and Jan Van Eyck, the proposed algorithm has been experimentally verified to outperform state-of-the-art X-ray separation methods in art investigation applications.