MPAN: Multi-Part Attention Network for Point Cloud Based 3D Shape Retrieval

Zirui Li,Junyu Xu,Wenhui Li,Weizhi Nie,Yue Zhao

doi:10.1109/access.2020.3018696

Zirui Li, Junyu Xu + Show 3 more

Open Access

https://doi.org/10.1109/access.2020.3018696

Copy DOI

Abstract

3D shape retrieval is an important researching field due to its wide applications in computer vision and multimedia fields. With the development of deep learning technology, great progress has been made in recent years and lots of methods have achieved promising 3D shape retrieval results. Due to the effective description of point cloud data on structural information for 3D shapes, lots of methods based on point cloud data format are proposed for better shape representation. However, most of them focus on extracting a global descrisptor from the whole 3D shape while the local features and detailed structural information are ignored, which negatively affect the effectiveness of shape descriptors. In addition, these methods also ignore the correlations among different parts of point clouds, which may introduce redundant information to the final shape descriptors. In order to address these issues, we propose a Multi-part attention network (MPAN) for 3D model retrieval based on point cloud. Firstly, we segment a 3D shape into multiple parts by employing a pre-trained PointNet++ segmentation model. After extracting the local features from them, we introduce a novel self-attention mechanism to explore the correlations between different parts. Meanwhile, by considering the structural relevance of them, the redundancy for representing 3D shapes is removed while the effective information is utilized. Finally, informative and discriminative shape descriptors, considering both local features and spatial correlations, are generated for 3D shape retrieval task. To validate the effectiveness of our method, we conduct several experiments on the public 3D shape benchmark, ShapeNetPart dataset. Experimental results and comparisons with state-of-the-art methods demonstrate the superiority of our proposed method.

Highlights

With the rapid development of computer vision and multimedia technologies, 3D shapes are widely utilized in a surge of fields such as virtual reality, 3D printing and industry designing
EVALUATION CRITERIA In the 3D shape retrieval task, several evaluation metrics are adopted to measure the performance of our proposed method, including Precision-Recall curve(PR-curve), Mean Average Precision(mAP), Nearest Neighbor(NN), the First Tier (FT), the Second Tier (ST), Discounted Cumulative Gain(NDCG) and Average Normalized modified retrieval rank (ANMRR)
In this paper, we propose a novel Multi-part Attention Network(MPAN) for 3D shape retrieval task, which utilizes the correlations between local features to strengthen the discrimination of 3D shapes

Summary

INTRODUCTION

With the rapid development of computer vision and multimedia technologies, 3D shapes are widely utilized in a surge of fields such as virtual reality, 3D printing and industry designing. PointNet++ employs a hierarchical feature extracting architecture for local features extraction and gradually expands the scope, ignoring the correlations of different regions These shortages have negative impacts on capturing structural information of point clouds, which limits the performance of shape descriptors generation and retrieval results. You et al [23] proposed a novel multi-modality method PVRNet, which fuses the features from two modalities by exploiting the correlations between point cloud and each individual view by the relevance This design fuses the two-modality data together and forms integrated shape descriptor, which effectively improve the shape retrieval performance. The correlations of multiple parts are important for describing the spatial structure of 3D shapes To address these problems, we propose a novel self-attention network for feature updating, considering the partial correlations and redundancy removing at the same time. The calculation process of ANMRR is elaborated in [31]

IMPLEMENTATION DETAILS

COMPARED WITH STATE-OF-ART METHODS

Findings

CONCLUSION