3D Image Annotation using Deep Learning and View-based Image Features

Abstract

The act of assigning word labels to images using machine learning algorithms is called automatic image annotation. Automatic annotation of image is used in various applications like media, medical, industrial and archaeological fields. Several methods have been proposed for automatic annotation of images, but most of them are focused on 2D images. In this article, we propose a new approach for 3D image annotation using deep learning and view-based image features. The most challenging issue in the automatic annotation of 3D images is to extract suitable features for image representation. 3D images are generally presented in the form of polygon meshes that are not suitable for deep learning. To counter the problem, we represent 3D images as several view-based images that are captured from different views. This process converts a 3D image into a multi-channel 2D image that can be classified using image-based deep classification networks. We utilized various classification networks for 3D image annotation, and the results showed the F1 score of 0.97 for the best architecture.