3D Retrieval Research Articles

Harmonic mean normalized Laplace–Beltrami spectral descriptor

This paper proposes a framework based on harmonic mean normalized Laplace---Beltrami spectral descriptor. A series of experiments show that the harmonic mean normalization has better performance for non-rigid 3D retrieval, and it is robust to holes, local scaling, noise and sampling. To better distinguish shapes with fine or rough details, weighting method and fusion method are also employed. Weighting method reduces the negative impact of high-frequency information, and fusion method combines multi-level spectral information in both low and high frequencies. Our approach has better performance than other state-of-the-art methods on both retrieval accuracy and time consumption for stretched non-rigid 3D shapes.

A New Efficient Optimal 2D Views Selection Method Based on Pivot Selection Techniques for 3D Indexing and Retrieval

In this paper, we propose a new method for 2D/3D object indexing and retrieval. The principle consists of an automatic selection of optimal views by using an incremental algorithm based on pivot selection techniques for proximity searching in metric spaces. The selected views are afterward described by four well-established descriptors from the MPEG-7 standard, namely: the color structure descriptor (CSD), the scalable color descriptor (SCD), the edge histogram descriptor (EHD) and the color layout descriptor (CLD). We present our results on two databases: The Amsterdam Library of Images (ALOI-1000), consisting of 72,000 color images of views, and the Columbia Object Image Library (COIL-100), consisting of 7200 color images of views. The results prove the performance of the developed method and its superiority over the k-means algorithm and the automatic selection of optimal views proposed by Mokhtarian et al.

An efficient and effective convolutional auto-encoder extreme learning machine network for 3d feature learning

3D shape features play a crucial role in graphics applications, such as 3D shape matching, recognition, and retrieval. Various 3D shape descriptors have been developed over the last two decades; however, existing descriptors are handcrafted features that are labor-intensively designed and cannot extract discriminative information for a large set of data. In this paper, we propose a rapid 3D feature learning method, namely, a convolutional auto-encoder extreme learning machine (CAE-ELM) that combines the advantages of the convolutional neuron network, auto-encoder, and extreme learning machine (ELM). This method performs better and faster than other methods. In addition, we define a novel architecture based on CAE-ELM. The architecture accepts two types of 3D shape representation, namely, voxel data and signed distance field data (SDF), as inputs to extract the global and local features of 3D shapes. Voxel data describe structural information, whereas SDF data contain details on 3D shapes. Moreover, the proposed CAE-ELM can be used in practical graphics applications, such as 3D shape completion. Experiments show that the features extracted by CAE-ELM are superior to existing hand-crafted features and other deep learning methods or ELM models. Moreover, the classification accuracy of the proposed architecture is superior to that of other methods on ModelNet10 (91.4%) and ModelNet40 (84.35%). The training process also runs faster than existing deep learning methods by approximately two orders of magnitude.

Novel Method for 3D Objects Retrieval

Access to appropriate information is a fundamental necessity in the modern society. Recent years have seen a rapid growth in use of digital images. To retrieve similar images from a database Content Based Image Retrieval (CBIR) system is used. Shape is most widely used feature in CBIR system. Among various approaches of shape retrieval, edge based shape descriptors are the most commonly been used. These representations are processed via different edge estimation tools and algorithms. However in the process of edge based coding, the discontinuous edge regions result in discarding of image regions or the inclusion of such regions increases the number of processing regions. A selection of non-informative region will lead to less descriptive feature, which leads to increase in system overhead and lowering of retrieval accuracy. Contours are observed to be the best representative approach for shape descriptor. To improve the retrieval accuracy of a 3D image retrieval system, a contour based coding is developed. In this approach along with the shape feature depth feature is extracted from the contor. The proposed 3D contor Shape depth (3D CSD) approach uses less number of feature vectors for representation. Simulation observations show that 3D CSD improves the retrieval accuracy and reduce the computation time.

Covariance-Based Descriptors for Efficient 3D Shape Matching, Retrieval, and Classification

State-of-the-art 3D shape classification and retrieval algorithms, hereinafter referred to as shape analysis, are often based on comparing signatures or descriptors that capture the main geometric and topological properties of 3D objects. None of the existing descriptors, however, achieve best performance on all shape classes. In this article, we explore, for the first time, the usage of covariance matrices of descriptors, instead of the descriptors themselves, in 3D shape analysis. Unlike histogram -based techniques, covariance-based 3D shape analysis enables the fusion and encoding of different types of features and modalities into a compact representation. Covariance matrices, however, are elements of the non-linear manifold of symmetric positive definite (SPD) matrices and thus ${\BBL_2}$ metrics are not suitable for their comparison and clustering. In this article, we study geodesic distances on the Riemannian manifold of SPD matrices and use them as metrics for 3D shape matching and recognition. We then: (1) introduce the concepts of bag of covariance (BoC) matrices and spatially-sensitive BoC as a generalization to the Riemannian manifold of SPD matrices of the traditional bag of features framework, and (2) generalize the standard kernel methods for supervised classification of 3D shapes to the space of covariance matrices. We evaluate the performance of the proposed BoC matrices framework and covariance -based kernel methods and demonstrate their superiority compared to their descriptor-based counterparts in various 3D shape matching, retrieval, and classification setups.

Local binary patterns on triangular meshes: Concept and applications

In this paper, we introduce an original framework for computing local binary like-patterns on 2D mesh manifolds (i.e., surfaces in the 3D space). This framework, dubbed mesh-LBP, preservers the simplicity and the adaptability of the 2D LBP and has the capacity of handling both open and close mesh surfaces without requiring normalization as compared to its 2D counterpart. We describe the foundations and the construction of mesh-LBP and showcase the different LBP patterns that can be generated on the mesh. In the experimentation, we provide evidence of the uniform patterns in the mesh-LBP, the repeatability of its descriptors, and its robustness to moderate shape deformations. Then, we show how the mesh-LBP descriptors can be adapted to a number of surface local and global analysis including 3D texture classification and retrieval, and 3D face matching. We also compare the performance of the mesh-LBP descriptors with a bunch of state of the art surface descriptors.

3D human motion retrieval using graph kernels based on adaptive graph construction

Graphs are frequently used to provide a powerful representation for structured data. However, it is still a challenging task to model 3D human motions due to its large spatio-temporal variations. This paper proposes a novel graph-based method for real time 3D human motion retrieval. Firstly, we propose a novel graph construction method which connects the joints that are deemed important for a given motion. In particular, the top-N Relative Ranges of Joint Relative Distances (RRJRD) were proposed to determine which joints should be connected in the resulting graph because these measures indicate the normalized activity levels among the joint pairs. Different motions may thus result in different graph structures so the construction of the graphs is made adaptive to the characteristics of a given motion and is able to represent a meaningful spatial structure. In addition to the spatial structure, the temporal pyramid of covariance descriptors was adopted to preserve certain level of spatio-temporal local features. The graph kernel is computed by matching the walks from each of the two graphs to be matched. Furthermore, multiple kernel learning was applied to determine the optimal weights for combining the graph kernels to measure the overall similarity between two motions. The experimental results show that our method is robust under several variations, and demonstrates superior performance in comparison to three state-of-the-art methods.

Multiscale shape context and re-ranking for deformable shape retrieval

This paper proposes a distinct multiscale shape context (MSC) method for isometric 3D shape description and retrieval. For each feature point, a MSC descriptor is devised to capture the multiple spatial information on the basis of the intrinsic shape context, which is advantageous in solving the domain offset deficiency for intra-class shapes. Different from the traditional shape context method, the MSC descriptor is built based on the charts without angular bins and the shape distributions in local domains, which makes it not only simple but also efficient. To reduce the cost of shape representation, we detect a sparse set of feature points and design an improved bags-of-word model to encode the MSC descriptors. For retrieval improvement, an efficient while robust re-ranking algorithm by metric mapping is designed to alleviate the errors of the feature space. Finally, the experimental results have demonstrated significant performance gains on two public benchmarks. Graphical abstractDisplay Omitted HighlightsA multiscale shape context (MSC) method is proposed for isometric 3D shape description and retrieval.A distinct bags-of-word model is designed based on MSC descriptors.A retrieval re-ranking algorithm based on metric mapping is devised.We report state-of-the-art performance for isometric shape retrieval.

3D object retrieval based on sparse coding in weak supervision

With the rapid development of computer vision and digital capture equipment, we can easily record the 3D information of objects. In the recent years, more and more 3D data are generated, which makes it desirable to develop effective 3D retrieval algorithms. In this paper, we apply the sparse coding method in a weakly supervision manner to address 3D model retrieval. First, each 3D object, which is represented by a set of 2D images, is used to learn dictionary. Then, sparse coding is used to compute the reconstruction residual for each query object. Finally, the residual between the query model and the candidate model is used for 3D model retrieval. In the experiment, ETH, NTU and ALOL dataset are used to evaluate the performance of the proposed method. The results demonstrate the superiority of the proposed method.

Evaluation of 3D Data Compression and Retrieval Method Based on Curve Mesh Filling

ABSTRACTDigital mock-up (DMU) and/or compound document with 3D data are one of the most important methods to represent a product model. Since the data size of 3D models becomes greater year by year, 3D data compression and retrieval algorithm is required to easily exchange such information through the Internet. Because broadband is still not popular in some regions in the world, it is necessary to send huge data in a small size as much as possible. Therefore, in this paper we developed an application system that transfers 3D surface models and then the performance of the 3D surface compression and retrieval algorithm is evaluated.

Salient views and view-dependent dictionaries for object recognition

A sparse representation-based approach is proposed to determine the salient views of 3D objects. The salient views are categorized into two groups. The first are boundary representative views that have several visible sides and object surfaces that may be attractive to humans. The second are side representative views that best represent views from sides of an approximating convex shape. The side representative views are class-specific and possess the most representative power compared to other within-class views. Using the concept of characteristic view class, we first present a sparse representation-based approach for estimating the boundary representative views. With the estimated boundaries, we determine the side representative views based on a minimum reconstruction error criterion. Furthermore, to evaluate our method, we introduce the notion of view-dependent dictionaries built from salient views for applications in 3D object recognition and retrieval. The proposed view-dependent dictionaries encode information on geometry across views and representation of the object. Through a series of experiments on four publicly available 3D object datasets, we demonstrate the effectiveness of our approach compared to two existing state-of-the-art algorithms and one baseline method.

Indirect shape analysis for 3D shape retrieval

We introduce indirect shape analysis, or ISA, where a given shape is analyzed not based on geometric or topological features computed directly from the shape itself, but by studying how external agents interact with the shape. The potential benefits of ISA are two-fold. First, agent–object interactions often reveal an object׳s function, which plays a key role in shape understanding. Second, compared to direct shape analysis, ISA, which utilizes pre-selected agents, is less affected by imperfections of, or inconsistencies between, the geometry or topology of the analyzed shapes. We employ digital human models as the external agents and develop a prototype ISA scheme for 3D shape classification and retrieval. Given a 3D model M, we compute an ISA feature vector for M by encoding how well a selected set of human models, with functional poses, can be aligned to M so as to perform the intended functions. We demonstrate the discriminability of ISA features for 3D shape retrieval and compare to state-of-the-art methods.

Multimodal Feature Fusion for 3D Shape Recognition and Retrieval

Three-dimensional shapes contain different kinds of information that jointly characterize the shape. Traditional methods, however, perform recognition or retrieval using only one type. This article presents a 3D feature learning framework that combines different modality data effectively to promote the discriminability of unimodal features. Two independent deep belief networks (DBNs) are employed to learn high-level features from low-level features, and a restricted Boltzmann machine (RBM) is trained for mining the deep correlations between the different modalities. Experiments demonstrate that the proposed method can achieve better performance.

Shift-invariant ring feature for 3D shape

In this paper, we present a shift-invariant ring feature for 3D shape, which can encode multiple low-level descriptors and provide high-discriminative representation of local region for 3D shape. First, several iso-geodesic rings are created at equal intervals, and then low-level descriptors on the sampling rings are used to represent the property of a feature point. In order to boost the descriptive capability of raw descriptors, we formulate the unsupervised basis learning into an L1-penalized optimization problem, which uses convolution operation to address the rotation ambiguity of descriptors resulting from different starting points in rings. In the following extraction procedure of high-level feature, we use the learned bases to calculate the sparse coefficients by solving the optimization problem. Furthermore, to make the coefficients irrelevant with the sequential order in ring, we use Fourier transform to achieve circular-shift invariant ring feature. Experiments on 3D shape correspondence and retrieval demonstrate the satisfactory performance of the proposed intrinsic feature.

Indexing 3D Scenes Using the Interaction Bisector Surface

The spatial relationship between different objects plays an important role in defining the context of scenes. Most previous 3D classification and retrieval methods take into account either the individual geometry of the objects or simple relationships between them such as the contacts or adjacencies. In this article we propose a new method for the classification and retrieval of 3D objects based on the Interaction Bisector Surface (IBS), a subset of the Voronoi diagram defined between objects. The IBS is a sophisticated representation that describes topological relationships such as whether an object is wrapped in, linked to, or tangled with others, as well as geometric relationships such as the distance between objects. We propose a hierarchical framework to index scenes by examining both the topological structure and the geometric attributes of the IBS. The topology-based indexing can compare spatial relations without being severely affected by local geometric details of the object. Geometric attributes can also be applied in comparing the precise way in which the objects are interacting with one another. Experimental results show that our method is effective at relationship classification and content-based relationship retrieval.

3D shape retrieval and classification using multiple kernel learning on extended Reeb graphs

We propose in this article a new 3D shape classification and retrieval method, based on a supervised selection of the most significant features in a space of attributed extended Reeb graphs encoding different shape characteristics. The similarity between pairs of graphs is addressed through both their representation as set of bags of shortest paths, and the definition of kernels adapted to these descriptions. A multiple kernel learning algorithm is used on this set of kernels to find an optimal linear combination of kernels for classification and retrieval purposes. Results on classical data sets are comparable with the best results of the literature, and the modularity and flexibility of the kernel learning ensure its applicability to a large set of methods.

New Technique for 3D Shape Retrieval in the Classified Databases

This study addresses the problem of 3D shape retrieval. While this problem is interesting and emerging as the size of 3D object databases grows rapidly, the main two issues the community has to focus on are: computational efficiency of 3D object retrieval and the quality of retrieval results. In this study we deal with the two considerations, especially the first one namely computational efficiency, by proposing a new technique to retrieve efficiently the 3D-objects in the classified databases which contains 3D objects of different categories. This technique can be coupled with any 3D retrieval method. In this study, we use the Clock Matching Bag-of-Features 3D retrieval method proposed by Lian et al. (2010) since it gives the best result comparing with several methods in particular the view based methods. Instead of systematically matching the object-query with all 3D objects of the target database, our approach restricts the pattern matching on a subset of good candidates (the most similar to the query). For a database classified in several classes the retrieval will be oriented to the right class that contains similar objects to the query. In this case, the matching process will be not systematically performed with all objects among the database, but only with objects of right class. Our key idea is to represent each class by one representative that will be used to orient the retrieval process to the right class. Experimental results illustrate the efficiency of our approach.

3D CAD Model Representation and Retrieval based on Hierarchical Graph

Due to 3D CAD models are often characterized with complicated geometry and topology, how to help the designers quickly and accurately find the object models from the database containing massive amount of models using the relatively rough query instance is still a big challenge. In this paper, a novel representation of 3D CAD models using hierarchical graph (HG) is proposed. The model descriptors are divided into shape feature descriptors and topology relationship descriptors. These two descriptors can be extracted from HG. In this way, coarse-grained and fine-grained 3D CAD model retrieval are implemented. On this basis, a 3D model retrieval method is proposed based on genetic algorithm (GA) and ant colony optimization (ACO), which are employed to detect the common sub-graph in the corresponding the hierarchical graphs of different models. This method can improve the accuracy and efficiency of 3D CAD model retrieval. Based on the above researches, a 3D model retrieval system HUST-CMRS is developed. Our experimental results show that, the algorithm proposed in this paper can implement multi-mode indexing and satisfy personalized need of users.

An Approach of Similarity Analysis for Non-Rigid Models Based on Diffusion Map with Thin-Plate Spline

Isometric transformation with geodesic distance is widely used in similarity analysis or 3D retrieval of non-rigid objects. However, geodesic distance is sensitive to topology noise and can be problematic in cases where objects topology is changed locally. In order to solve the problem, a new method based on diffusion map was proposed. We used diffusion map to compute embedded models and utilized thin-plate spline (TPS) to match the embedded models to obtain the similarity error of original objects. The method was tested on some examples, the results show that the new method is suitable to similarity analysis or 3D retrieval of non-rigid objects, moreover, our method is robust to the deformed objects with topology noise.

Fully interferometric three-dimensional imaging spectrometry using hyperbolic-type volume interferogram

A signal-processing method is proposed in the fully interferometric three-dimensional (3D) imaging spectrometry. This processing computes a 3D interferogram, in which recorded fringe patterns do not directly reflect wavefront forms propagated from a polychromatic light source under measurement. This paper presents a procedure for signal processing including a synthesis of the 3D interferogram and retrieval of a set of spectral components of 3D images. We demonstrate retrieving 3D images for spectral components of two planar light sources by means of the proposed method. The procedure to synthesize the 3D interferogram in this method suggests the possibility of direct measurement of the 3D interferogram.