Multi-scale and multi-level shape descriptor learning via a hybrid fusion network

Abstract

Discriminative and informative 3D shape descriptors are of fundamental significance to computer graphics applications, especially in the fields of geometry modeling and shape analysis. 3D shape descriptors, which reveal extrinsic/intrinsic properties of 3D shapes, have been well studied for decades and proved to be useful and effective in various analysis and synthesis tasks. Nonetheless, existing descriptors are mainly founded upon certain local differential attributes or global shape spectra, and certain combinations of both types. Conventional descriptors are typically customized for specific tasks with priori domain knowledge, which severely prevents their applications from widespread use. Recently, neural networks, benefiting from their powerful data-driven capability for general feature extraction from raw data without any domain knowledge, have achieved great success in many areas including shape analysis. In this paper, we present a novel hybrid fusion network (HFN) that learns multi-scale and multi-level shape representations via uniformly integrating a traditional region-based descriptor with modern neural networks. On one hand, we exploit the spectral graph wavelets (SGWs) to extract the shapes’ local-to-global features. On the other hand, the shapes are fed into a convolutional neural network to generate multi-level features simultaneously. Then a hierarchical fusion network learns a general and unified representation from these two different types of features which capture multi-scale and multi-level properties of the underlying shapes. Extensive experiments and comprehensive comparisons demonstrate our HFN can achieve better performance in common shape analysis tasks, such as shape retrieval and recognition, and the learned hybrid descriptor is robust, informative, and discriminative with more potential for widespread applications.