3D Shape Retrieval Methods Research Articles

Soft Margin Triplet-Center Loss for Multi-View 3D Shape Retrieval

Obtaining discriminative features is one of the key problems in three-dimensional (3D) shape retrieval. Recently, deep metric learning-based 3D shape retrieval methods have attracted the researchers’ attention and have achieved better performance. The triplet-center loss can learn more discriminative features than traditional classification loss, and it has been successfully used in deep metric learning-based 3D shape retrieval task. However, it has a hard margin parameter that only leverages part of the training data in each mini-batch. Moreover, the margin parameter is often determined by experience and remains unchanged during the training process. To overcome the above limitations, we propose the soft margin triplet-center loss, which replaces the margin with the nonparametric soft margin. Furthermore, we combined the proposed soft margin triplet-center loss with the softmax loss to improve the training efficiency and the retrieval performance. Extensive experimental results on two popular 3D shape retrieval datasets have validated the effectiveness of the soft margin triplet-center loss, and our proposed 3D shape retrieval method has achieved better performance than other state-of-the-art method.

A method of generating depth images for view-based shape retrieval of 3D CAD models from partial point clouds

Laser scanners can easily acquire the geometric data of physical environments in the form of point clouds. Industrial 3D reconstruction processes generally recognize objects from point clouds, which should include both geometric and semantic data. However, the recognition process is often a bottleneck in 3D reconstruction because it is labor intensive and requires expertise in domain knowledge. To address this problem, various methods have been developed to recognize objects by retrieving their corresponding models from a database via input geometric queries. In recent years, geometric data conversion to images and view-based 3D shape retrieval applications have demonstrated high accuracies. Depth images that encode the depth values as pixel intensities are frequently used for view-based 3D shape retrieval. However, geometric data collected from objects are often incomplete owing to occlusions and line-of-sight limitations. Images generated by occluded point clouds lower the view-based 3D object retrieval performance owing to loss of information. In this paper, we propose a viewpoint and image-resolution estimation method for view-based 3D shape retrieval from point cloud queries. Further, automatic selection of viewpoint and image resolution are proposed using the data acquisition rate and density calculations from sampled viewpoints and image resolutions. The retrieval performances for images generated by the proposed method are investigated via experiments and compared for various datasets. Additionally, view-based 3D shape retrieval performance with a deep convolutional neural network was investigated using the proposed method.

A comparison of methods for 3D scene shape retrieval

3D scene shape retrieval is a brand new but important research direction in content-based 3D shape retrieval. To promote this research area, two Shape Retrieval Contest (SHREC) tracks on 2D scene sketch-based and image-based 3D scene model retrieval have been organized by us in 2018 and 2019, respectively. In 2018, we built the first benchmark for each track which contains 2D and 3D scene data for ten (10) categories, while they share the same 3D scene target dataset. Four and five distinct 3D scene shape retrieval methods have competed with each other in these two contests, respectively. In 2019, to measure and compare the scalability performance of the participating and other promising Query-by-Sketch or Query-by-Image 3D scene shape retrieval methods, we built a much larger extended benchmark for each type of retrieval which has thirty (30) classes and organized two extended tracks. Again, two and three different 3D scene shape retrieval methods have contended in these two tracks, separately. To solicit state-of-the-art approaches, we perform a comprehensive comparison of all the above methods and an additional new retrieval methods by evaluating them on the two benchmarks. The benchmarks, evaluation results and tools are publicly available at our track websites (Yuan et al., 2019 [1]; Abdul-Rashid et al., 2019 [2]; Yuan et al., 2019 [3]; Abdul-Rashid et al., 2019 [4]), while code for the evaluated methods are also available: http://github.com/3DSceneRetrieval.

Non-rigid 3D shape retrieval based on multi-scale graphical image and joint Bayesian

Feature analysis plays a crucial role in various applications in both computer vision and computer graphics. The semantic gap between 2D images and 3D graphical models is the major obstacle to improve the universality of existing valuable technologies. To bridge the gap, we propose an effective and robust representation of 3D models, named multi-scale Graphical Image (GI), which is constructed by introducing the statistics mapping from 3D models to 2D images with both local and global information. Therefore, the excellent innovations and techniques in 2D visual retrieval can be adapted to 3D geometric retrieval. In the multi-scale GI space, the joint Bayesian formulation is exploited to analyze the structure of the space and learn a new metric. It benefits several attractive properties, including high discriminative, isometric invariant and robust to noise and topological changes, etc. In order to prove the validity, we apply the proposed method to 3D shape retrieval, and test our method on two well-known benchmark datasets. The results show that our method substantially outperforms the state-of-the-art non-rigid 3D shape retrieval methods.

Angular Triplet-Center Loss for Multi-View 3D Shape Retrieval

How to obtain the desirable representation of a 3D shape, which is discriminative across categories and polymerized within classes, is a significant challenge in 3D shape retrieval. Most existing 3D shape retrieval methods focus on capturing strong discriminative shape representation with softmax loss for the classification task, while the shape feature learning with metric loss is neglected for 3D shape retrieval. In this paper, we address this problem based on the intuition that the cosine distance of shape embeddings should be close enough within the same class and far away across categories. Since most of 3D shape retrieval tasks use cosine distance of shape features for measuring shape similarity, we propose a novel metric loss named angular triplet-center loss, which directly optimizes the cosine distances between the features. It inherits the triplet-center loss property to achieve larger inter-class distance and smaller intra-class distance simultaneously. Unlike previous metric loss utilized in 3D shape retrieval methods, where Euclidean distance is adopted and the margin design is difficult, the proposed method is more convenient to train feature embeddings and more suitable for 3D shape retrieval. Moreover, the angle margin is adopted to replace the cosine margin in order to provide more explicit discriminative constraints on an embedding space. Extensive experimental results on two popular 3D object retrieval benchmarks, ModelNet40 and ShapeNetCore 55, demonstrate the effectiveness of our proposed loss, and our method has achieved state-ofthe-art results on various 3D shape datasets.

Multi-feature distance metric learning for non-rigid 3D shape retrieval

In the past decades, feature-learning-based 3D shape retrieval approaches have been received widespread attention in the computer graphic community. These approaches usually explored the hand-crafted distance metric or conventional distance metric learning methods to compute the similarity of the single feature. The single feature always contains onefold geometric information, which cannot characterize the 3D shapes well. Therefore, the multiple features should be used for the retrieval task to overcome the limitation of single feature and further improve the performance. However, most conventional distance metric learning methods fail to integrate the complementary information from multiple features to construct the distance metric. To address these issue, a novel multi-feature distance metric learning method for non-rigid 3D shape retrieval is presented in this study, which can make full use of the complimentary geometric information from multiple shape features by utilizing the KL-divergences. Minimizing KL-divergence between different metric of features and a common metric is a consistency constraints, which can lead the consistency shared latent feature space of the multiple features. We apply the proposed method to 3D model retrieval, and test our method on well known benchmark database. The results show that our method substantially outperforms the state-of-the-art non-rigid 3D shape retrieval methods.

Deeply Exploiting Long-Term View Dependency for 3D Shape Recognition

Recognition of 3D shapes is a fundamental task in computer vision. In recent years, view-based deep learning has emerged as an effective approach for 3D shape recognition. Most existing view-based methods treat the views of an object as an unordered set, which ignores the dynamic relations among the views, e.g. sequential semantic dependencies. In this paper, modeling the views of an object by a sequence, we aim at exploiting the long-term dependencies among different views for shape recognition, which is done by constructing a sequence-aware view aggregation module based on the bi-directional Long Short-Term Memory network. It is shown that our view aggregation module not only captures the bi-directional dependencies in view sequences, but also enjoys the robustness to circular shifts of input sequences. Incorporating the aggregation module into a standard convolutional network architecture, we develop an effective method for 3D shape classification and retrieval. Our method was evaluated on the ModelNet40/10 and ShapeNetCore55 datasets. The results show the encouraging performance gain from exploiting long-term dependencies in view sequences, as well as the superior performance of our method compared to the existing ones.

Local commute-time guided MDS for 3D non-rigid object retrieval

In this paper, we propose a 3D non-rigid shape retrieval method based on canonical shape analysis. Our main idea is to transform the problem of non-rigid shape retrieval into a rigid shape retrieval problem via the well-known multidimensional scaling (MDS) approach and random walk on graphs. We first segment the non-rigid shape into local partitions based on its salient features. Then, we calculate a local MDS problem for each partition, where the local commute time distance is used as weighting function in order to preserve local shape details. Finally, we aggregate the set of local MDS problems as a global constrained problem. The constraint is formulated using the biharmonic function between local salient features. In contrast to MDS method, the proposed local MDS is computationally efficient, parameters free and gives isometry-invariant forms with minimum features distortion. Due to these advantageous properties, the proposed method achieved good retrieval accuracy on non-rigid shape benchmark datasets.

Deep Multimetric Learning for Shape-Based 3D Model Retrieval

Recently, feature-learning-based 3D shape retrieval methods have been receiving more and more attention in the 3D shape analysis community. In these methods, the hand-crafted metrics or the learned linear metrics are usually used to compute the distances between shape features. Since there are complex geometric structural variations with 3D shapes, the single hand-crafted metric or learned linear metric cannot characterize the manifold, where 3D shapes lie well. In this paper, by exploring the nonlinearity of the deep neural network and the complementarity among multiple shape features, we propose a novel deep multimetric network for 3D shape retrieval. The developed multimetric network minimizes a discriminative loss function that, for each type of shape feature, the outputs of the network from the same class are encouraged to be as similar as possible and the outputs from different classes are encouraged to be as dissimilar as possible. Meanwhile, the Hilbert-Schmidt independence criterion is employed to enforce the outputs of different types of shape features to be as complementary as possible. Furthermore, the weights of the learned multiple distance metrics can be adaptively determined in our developed deep metric network. The weighted distance metric is then used as the similarity for shape retrieval. We conduct experiments with the proposed method on the four benchmark shape datasets. Experimental results demonstrate that the proposed method can obtain better performance than the learned deep single metric and outperform the state-of-the-art 3D shape retrieval methods.

Sketch-Based Articulated 3D Shape Retrieval.

Sketch-based queries are a suitable and superior alternative to traditional text- and example-based queries for 3D shape retrieval. The authors developed an articulated 3D shape retrieval method that uses easy-to-obtain 2D sketches. In contrast to existing sketch-based retrieval systems that lower the 3D database models to 2D, their algorithm implicitly lifts the 2D query to 2.5D by inferring depth information from possibly self-intersecting sketches using a good continuation rule. It does not require 3D example models to initiate queries, but results show that it achieves accuracy comparable to a state-of-the-art example-based 3D shape retrieval method.

2D compressive sensing and multi-feature fusion for effective 3D shape retrieval

3D shape retrieval is always a challenging research topic because of complex geometric structural variations involved. Although many feature extraction and retrieval algorithms have been proposed, they generally only use single 3D model descriptor hence cannot obtain better retrieval performance. In this paper, we propose a new 3D shape retrieval framework based on compressive sensing (CS) and multi-feature fusion (MFF). Firstly, we extract three new features including the CS Chebyshev ray (CSCR) feature, the CS spatial hierarchical (CSSH) feature and the Extended Gaussian sphere (EGS) feature. Actually, CSCR, CSSH and EGS respectively represent the volume tensor, the layered detail and the statistical space distribution on the model surface of a 3D model. To make the best use of these features, a supervised learning is used to determine the weighting coefficients for these features. Finally, the features and their corresponding weighting coefficients are used to determine the similarity of 3D models in the multi-feature fusion (MFF) framework for 3D shape retrieval. For performance assessment, two publicly available datasets that contain 3D models with large geometric variations are used, including ModelNet-10 and PSB datasets. Comprehensive experimental results have demonstrated the efficacy of the proposed method for 3D shape retrieval.

DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval.

Complex geometric variations of 3D models usually pose great challenges in 3D shape matching and retrieval. In this paper, we propose a novel 3D shape feature learning method to extract high-level shape features that are insensitive to geometric deformations of shapes. Our method uses a discriminative deep auto-encoder to learn deformation-invariant shape features. First, a multiscale shape distribution is computed and used as input to the auto-encoder. We then impose the Fisher discrimination criterion on the neurons in the hidden layer to develop a deep discriminative auto-encoder. Finally, the outputs from the hidden layers of the discriminative auto-encoders at different scales are concatenated to form the shape descriptor. The proposed method is evaluated on four benchmark datasets that contain 3D models with large geometric variations: McGill, SHREC'10 ShapeGoogle, SHREC'14 Human and SHREC'14 Large Scale Comprehensive Retrieval Track Benchmark datasets. Experimental results on the benchmark datasets demonstrate the effectiveness of the proposed method for 3D shape retrieval.

A 3D shape retrieval method for orthogonal fringe projection based on a combination of variational image decomposition and variational mode decomposition

The orthogonal fringe projection technique has as wide as long practical application nowadays. In this paper, we propose a 3D shape retrieval method for orthogonal composite fringe projection based on a combination of variational image decomposition (VID) and variational mode decomposition (VMD). We propose a new image decomposition model to extract the orthogonal fringe. Then we introduce the VMD method to separate the horizontal and vertical fringe from the orthogonal fringe. Lastly, the 3D shape information is obtained by the differential 3D shape retrieval method (D3D). We test the proposed method on a simulated pattern and two actual objects with edges or abrupt changes in height, and compare with the recent, related and advanced differential 3D shape retrieval method (D3D) in terms of both quantitative evaluation and visual quality. The experimental results have demonstrated the validity of the proposed method.

A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries

Large-scale 3D shape retrieval has become an important research direction in content-based 3D shape retrieval. To promote this research area, two Shape Retrieval Contest (SHREC) tracks on large scale comprehensive and sketch-based 3D model retrieval have been organized by us in 2014. Both tracks were based on a unified large-scale benchmark that supports multimodal queries (3D models and sketches). This benchmark contains 13680 sketches and 8987 3D models, divided into 171 distinct classes. It was compiled to be a superset of existing benchmarks and presents a new challenge to retrieval methods as it comprises generic models as well as domain-specific model types. Twelve and six distinct 3D shape retrieval methods have competed with each other in these two contests, respectively. To measure and compare the performance of the participating and other promising Query-by-Model or Query-by-Sketch 3D shape retrieval methods and to solicit state-of-the-art approaches, we perform a more comprehensive comparison of twenty-six (eighteen originally participating algorithms and eight additional state-of-the-art or new) retrieval methods by evaluating them on the common benchmark. The benchmark, results, and evaluation tools are publicly available at our websites (http://www.itl.nist.gov/iad/vug/sharp/contest/2014/Generic3D/, 2014, http://www.itl.nist.gov/iad/vug/sharp/contest/2014/SBR/, 2014).

A comparison of methods for sketch-based 3D shape retrieval

Sketch-based 3D shape retrieval has become an important research topic in content-based 3D object retrieval. To foster this research area, two Shape Retrieval Contest (SHREC) tracks on this topic have been organized by us in 2012 and 2013 based on a small-scale and large-scale benchmarks, respectively. Six and five (nine in total) distinct sketch-based 3D shape retrieval methods have competed each other in these two contests, respectively. To measure and compare the performance of the top participating and other existing promising sketch-based 3D shape retrieval methods and solicit the state-of-the-art approaches, we perform a more comprehensive comparison of fifteen best (four top participating algorithms and eleven additional state-of-the-art methods) retrieval methods by completing the evaluation of each method on both benchmarks. The benchmarks, results, and evaluation tools for the two tracks are publicly available on our websites [1,2].

CM-BOF: visual similarity-based 3D shape retrieval using Clock Matching and Bag-of-Features

Content-based 3D object retrieval has become an active topic in many research communities. In this paper, we propose a novel visual similarity-based 3D shape retrieval method (CM-BOF) using Clock Matching and Bag-of-Features. Specifically, pose normalization is first applied to each object to generate its canonical pose, and then the normalized object is represented by a set of depth-buffer images captured on the vertices of a given geodesic sphere. Afterwards, each image is described as a word histogram obtained by the vector quantization of the image's salient local features. Finally, an efficient multi-view shape matching scheme (i.e., Clock Matching) is employed to measure the dissimilarity between two models. When applying the CM-BOF method in non-rigid 3D shape retrieval, multidimensional scaling (MDS) should be utilized before pose normalization to calculate the canonical form for each object. This paper also investigates several critical issues for the CM-BOF method, including the influence of the number of views, codebook, training data, and distance function. Experimental results on five commonly used benchmarks demonstrate that: (1) In contrast to the traditional Bag-of-Features, the time-consuming clustering is not necessary for the codebook construction of the CM-BOF approach; (2) Our methods are superior or comparable to the state of the art in applications of both rigid and non-rigid 3D shape retrieval.

A comparison of methods for non-rigid 3D shape retrieval

Non-rigid 3D shape retrieval has become an active and important research topic in content-based 3D object retrieval. The aim of this paper is to measure and compare the performance of state-of-the-art methods for non-rigid 3D shape retrieval. The paper develops a new benchmark consisting of 600 non-rigid 3D watertight meshes, which are equally classified into 30 categories, to carry out experiments for 11 different algorithms, whose retrieval accuracies are evaluated using six commonly utilized measures. Models and evaluation tools of the new benchmark are publicly available on our web site [1].

3D Shape and Structure Retrieval Method of Mechanical Parts Based on Recursive Segmentation

To realize reusable design, the retrieval of existing engineering design cases is necessary in which the most difficult problem is shape similarity comparison. Therefore, a 3D shape and structure retrieval method of mechanical parts based on recursive segmentation is put forward. The initial position of mechanical parts is obtained by generating convex bounding box of the mechanical parts and the normalization transformation of mechanical parts is realized. The solid model is segmented by generating adaptive bounded segmentation plane. The ordered full binary tree is built through recursive algorithm with the change detection of both the geometry and topology of the model. The feature vectors combined with engineer features are constructed to obtain the similarity comparison of the solid model in non-root nodes by means of similarity comparison of the vectors. The breath-first and variable weight traversal of the ordered full binary tree is realized. The similarity of 3D shape and structure between the mechanical parts is obtained via the similarity comparison of the solid model in non-root nodes. The application and verification of 3D shape and structure retrieval method of mechanical parts based on recursive segmentation in prototype system indicates that the method has higher calculation efficiency and judgment precision and it can reflect not only the global sketchy shape but also the local detail structure of the mechanical parts. It has important significance to realizing design reuse and improving design efficiency.

Multi-Fourier spectra descriptor and augmentation with spectral clustering for 3D shape retrieval

We propose a new method of similarity search for 3D shape models, given an arbitrary 3D shape as a query. The method features the high search performance enabled in part by our unique feature vector called Multi-Fourier Spectra Descriptor (MFSD), and in part by augmenting the feature vector with spectral clustering. The MFSD is composed of four independent Fourier spectra with periphery enhancement. It allows us to faithfully capture the inherent characteristics of an arbitrary 3D shape object regardless of the dimension, orientation, and original location of the object when it is first defined. Given a 3D shape database, the augmentation with spectral clustering is done first by computing the p-minimum spanning tree of the whole data set, where p is a number usually much less than m, the size of the whole 3D shape data set. We then define the affinity matrix, which is a square matrix of size m by m, where each element of the matrix denotes the distance between two shape objects. The distance is computed in advance by traversing the p-minimum spanning tree. The eigenvalue decomposition is then applied to the affinity matrix to reduce dimensionality of the matrix, followed by grouping into k clusters. The cluster information is kept for augmenting the search performance when a query is given. With a series of benchmark data sets, we will demonstrate that our approach outperforms previously known methods for 3D shape retrieval.

A survey of content based 3D shape retrieval methods

Recent developments in techniques for modeling, digitizing and visualizing 3D shapes has led to an explosion in the number of available 3D models on the Internet and in domain-specific databases. This has led to the development of 3D shape retrieval systems that, given a query object, retrieve similar 3D objects. For visualization, 3D shapes are often represented as a surface, in particular polygonal meshes, for example in VRML format. Often these models contain holes, intersecting polygons, are not manifold, and do not enclose a volume unambiguously. On the contrary, 3D volume models, such as solid models produced by CAD systems, or voxels models, enclose a volume properly. This paper surveys the literature on methods for content based 3D retrieval, taking into account the applicability to surface models as well as to volume models. The methods are evaluated with respect to several requirements of content based 3D shape retrieval, such as: (1) shape representation requirements, (2) properties of dissimilarity measures, (3) efficiency, (4) discrimination abilities, (5) ability to perform partial matching, (6) robustness, and (7) necessity of pose normalization. Finally, the advantages and limitations of the several approaches in content based 3D shape retrieval are discussed.