A hybrid 3D object auto-completion approach with self-supervised data augmentation for fragments of archaeological objects

Abstract

It is a common case that archaeological finds are frequently found in fragments and cannot always be reassembled in their entirety. Additionally, the completion of their shape and volume is sometimes costly and occasionally requires a lot of manual work, in particular for exhibition purposes. As a first step, improving the geometry of the fragments could potentially help the archaeological conservators in the assembling process. In this direction, in this paper we propose an end-to-end hybrid approach (i.e. point clouds to benefit from the limited complexity and creating results in mesh benefiting from their visualization capabilities) for the digital auto-completion of fragmented archaeological artifacts with missing parts in combination with a novel self-supervised data augmentation scheme in order to address a significant limitation in the archaeological domain (i.e. limited available observations and absence of the complete object). Our approach uses only 5 fragments of a partly preserved archaeological object for training. As a first step, an object (i.e artifact fragment) in mesh representation is converted to point cloud, from which more fragments of the incomplete original object are artificially generated, with the help of 3D surfaces, to augment the dataset. This dataset is then passed in a GAN-based Cascaded refinement network for generating a complete output, which is, finally, converted back to a mesh. Our experiments showed that the proposed hybrid approach, can be successfully used to complete and restore the geometry of a partly preserved archaeological object without having ever seen the complete form of it. However, due to the complexity of the fragments, such methods in most cases cannot generate their fine decorative and surface details.