Analyzing part functionality via multi-modal latent space embedding and interweaving

Abstract

In this paper, we advocate a novel method for analyzing the functionality of parts in 3D objects. In contrast to prior research, our method no longer characterizes the functionality of an object part using its single type of qualities (or attributes), e.g., geometry or interactions, nor by weighing the significance of various qualities. Instead, we consider the latent space of part functions as a semantic feature space comprehensively defined by part qualities. To learn such a space by parameterizing and encoding semantic features from multi-channel, we begin by learning multi-modal latent space using shapes, textures, and interaction scenes. Next, the latent space of part functions is generated by embedding and interweaving these multi-modal spaces into a space with a higher dimension. We devise loss functions to direct the embedding and interweaving of multi-modal spaces while preserving their manifolds. Consequently, the learned functionality latent space can capture the similarities between semantic features related to functionality and encode them into high-level functional representations. We assess this innovative approach on diverse categories of textured 3D shapes. Extensive experiments have exhibited our method’s parametric and encoding capability towards functionality-centric shape analysis and synthesis, including shape functionality analysis, functionally-similar shape retrieval, and functionality-aware modeling, all of which are of the essence to new graphics techniques and applications.