3D Object Retrieval Research Articles

Enhanced pose normalization and matching of non-rigid objects based on support vector machine modelling

The estimation of 3D surface correspondence constitutes a fundamental problem in shape matching and analysis applications. In the presence of non-rigid shape deformations, the ambiguity of surface correspondence increases together with the complexity of registration algorithms.In this paper, we alleviate this problem by means of 3D pose normalization using One-Class Support Vector Machines (OCSVM). In detail, we show how OCSVM are employed in order to increase the consistency of translation and scale normalization under articulations, extrusions or the presence of outliers. To estimate the relative translation and scale of an object, we use the 3D distribution of points that is modelled by employing OCSVM to estimate the decision surface corresponding to the surface points of the object under a preset tolerance to outliers. By discarding the outliers in the estimation of the object's center and size we compute the canonical pose of the core distribution that is less sensitive to intra-class shape variations. The effectiveness of the proposed method is demonstrated through the increased stability of translation and scale normalization and further justified by improving the precision of content-based 3D object retrieval.

Generic fitted shapes (GFS): Volumetric object segmentation in service robotics

In this paper, a simultaneous 3D volumetric segmentation and reconstruction method, based on the so-called Generic Fitted Shapes (GFS) is proposed. The aim of this work is to cope with the lack of volumetric information encountered in visually controlled mobile manipulation systems equipped with stereo or RGB-D cameras. Instead of using primitive volumes, such as cuboids or cylinders, for approximating objects in point clouds, their volumetric structure has been estimated based on fitted generic shapes. The proposed GFSs can capture the shapes of a broad range of object classes without the need of large a-priori shape databases. The fitting algorithm, which aims at determining the particular geometry of each object of interest, is based on a modified version of the active contours approach extended to the 3D Cartesian space. The proposed volumetric segmentation system produces comprehensive closed object surfaces which can be further used in mobile manipulation scenarios. Within the experimental setup, the proposed technique has been evaluated against two state-of-the-art methods, namely superquadrics and 3D Object Retrieval (3DOR) engines.

An Extension of 3D Zernike Moments for Shape Description and Retrieval of Maps Defined in Rectangular Solids

Abstract Zernike polynomials have been widely used in the description and shape retrieval of 3D objects. These orthonormal polynomials allow for efficient description and reconstruction of objects that can be scaled to fit within the unit ball. However, maps defined within box-shaped regions ¶ for example, rectangular prisms or cubes ¶ are not well suited to representation by Zernike polynomials, because these functions are not orthogonal over such regions. In particular, the representations require many expansion terms to describe object features along the edges and corners of the region. We overcome this problem by applying a Gram-Schmidt process to re-orthogonalize the Zernike polynomials so that they recover the orthonormality property over a specified box-shaped domain. We compare the shape retrieval performance of these new polynomial bases to that of the classical Zernike unit-ball polynomials.

CM-BOF: visual similarity-based 3D shape retrieval using Clock Matching and Bag-of-Features

Content-based 3D object retrieval has become an active topic in many research communities. In this paper, we propose a novel visual similarity-based 3D shape retrieval method (CM-BOF) using Clock Matching and Bag-of-Features. Specifically, pose normalization is first applied to each object to generate its canonical pose, and then the normalized object is represented by a set of depth-buffer images captured on the vertices of a given geodesic sphere. Afterwards, each image is described as a word histogram obtained by the vector quantization of the image's salient local features. Finally, an efficient multi-view shape matching scheme (i.e., Clock Matching) is employed to measure the dissimilarity between two models. When applying the CM-BOF method in non-rigid 3D shape retrieval, multidimensional scaling (MDS) should be utilized before pose normalization to calculate the canonical form for each object. This paper also investigates several critical issues for the CM-BOF method, including the influence of the number of views, codebook, training data, and distance function. Experimental results on five commonly used benchmarks demonstrate that: (1) In contrast to the traditional Bag-of-Features, the time-consuming clustering is not necessary for the codebook construction of the CM-BOF approach; (2) Our methods are superior or comparable to the state of the art in applications of both rigid and non-rigid 3D shape retrieval.

Best papers in multimedia information retrieval

An important area in multimedia information retrieval is event recognition in video retrieval. This area is of particular interest because it directly addresses the grand challenge of bridging the semantic gap between the user and the machine. We open this issue with the survey paper “High-level event recognition in unconstrained videos” by Yu-Gang Jiang, Subhabrata Bhattacharya, Shih-Fu Chang and Mubarak Shah, which presents a comprehensive review, breakdown and analysis of the current and critical technologies in state-of-the-art high-level event recognition systems. On behalf of the editorial board, it is an honor to present extended versions of the four best conference papers from the ACM multimedia retrieval research community. The papers were selected based on the reviews of the program committee by the ACM ICMR program chairs, Alan Smeaton, Alex Hauptmann, and Chong-Wah Ngo and the ACM MIR program chairs, Nuria Ramirez and Apostol Natsev. Due to the prevalence of 3D objects in architecture, CAD applications, computer games, and numerous internet applications, 3D object retrieval has grown quickly in societal importance. The paper “3D object retrieval using salient views” by Indriyati Atmosukarto and Linda G. Shapiro presents a novel approach based on silhouettes which is both competitive in accuracy with the top research community methods and also significantly higher performance.

Learning Semantic Signatures for 3D Object Retrieval

In this paper, we propose two kinds of semantic signatures for 3D object retrieval (3DOR). Humans are capable of describing an object using attribute terms like “symmetric” and “flyable”, or using its similarities to some known object classes. We convert such qualitative descriptions into attribute signature (AS) and reference set signature (RSS), respectively, and use them for 3DOR. We also show that AS and RSS can be understood as two different quantization methods of the same semantic space of human descriptions of objects. The advantages of the semantic signatures are threefold. First, they are much more compact than low-level shape features yet working with comparable retrieval accuracy. Therefore, the proposed semantic signatures require less storage space and computation cost in retrieval. Second, the high-level signatures are a good complement to low-level shape features. As a result, by incorporating the signatures we can improve the performance of state-of-the-art 3DOR methods by a large margin. To the best of our knowledge, we obtain the best results on two popular benchmarks. Third, the AS enables us to build a user-friendly interface, with which the user can trigger a search by simply clicking attribute bars instead of finding a 3D object as the query. This interface is of great significance in 3DOR considering the fact that while searching, the user usually does not have a 3D query at hand that is similar to his/her targeted objects in the database.

3D Object Retrieval Based on PSO-K-Modes Method

By use of semantic attributes of 3D object, the user can search for targeted objects, which main advantage is that it does not require the user to sketch a 3D object as the query for 3D object retrieval, and the retrieval system can obtain a better retrieval performance. There are many categorical datum among these attributes, and how to use those and find the most similar objects is a vital problem to resolve. However, several elements with different types may have a shorter Euclidean distance. It is obvious the objects belonging to the same category are closer. Therefore, we present a 3D object retrieval method with clustering principle and RBF interpolator, which need a robust clustering method. The k-modes is a classic clustering algorithm for categorical data set. Its principle is simple, but it is easy to converge to a local optimum. PSO (Particle Swarm Optimization) algorithm is an effective tool for optimization, so we attempt to overcome the local optimum problem with PSO for categorical data set. PSO usually used to solve continuous optimization problems., but the categorical data are non-continuous. This paper presents an a novel k-p-modes algorithm to overcome these problems. Results show the method is effective.

多視点画像の局所的な特徴の相関に基づく三次元物体の形状類似検索

近年，スマートフォンの普及や Web ブラウザの機能向上により，三次元物体による豊かな表現を利用したアプリケーションやサービスが増加している．今後，文書・画像データのように，Web 上に三次元物体データが爆発的に増加することが予想される．これにともない，大量にある三次元物体から，形状的特徴を手がかりに検索を行う，三次元物体の形状類似検索技術が必要とされている．本論文では，三次元物体から生成したDepth Buffer画像の局所特徴量の相関による，新しい三次元物体の特徴量Local Feature Correlation Descriptor（LCoD）を提案する．LCoD では，まず，複数視点よりレンダリングした Depth Buffer画像から局所特徴量を抽出する．そして，各 Depth Buffer 画像ごとに，局所特徴量の相関行列を求め，相関行列の要素を並べたものを，三次元物体の特徴量とする．LCoD は，Princeton Shape Benchmark を用いた比較実験において，検索結果上位での適合率を表すFirst Tier が 0.4708 となり，Light-Field Descriptorなどの従来手法よりも優れた検索性能を示した．

View-Based Discriminative Probabilistic Modeling for 3D Object Retrieval and Recognition

In view-based 3D object retrieval and recognition, each object is described by multiple views. A central problem is how to estimate the distance between two objects. Most conventional methods integrate the distances of view pairs across two objects as an estimation of their distance. In this paper, we propose a discriminative probabilistic object modeling approach. It builds probabilistic models for each object based on the distribution of its views, and the distance between two objects is defined as the upper bound of the Kullback-Leibler divergence of the corresponding probabilistic models. 3D object retrieval and recognition is accomplished based on the distance measures. We first learn models for each object by the adaptation from a set of global models with a maximum likelihood principle. A further adaption step is then performed to enhance the discriminative ability of the models. We conduct experiments on the ETH 3D object dataset, the National Taiwan University 3D model dataset, and the Princeton Shape Benchmark. We compare our approach with different methods, and experimental results demonstrate the superiority of our approach.

A comparison of methods for non-rigid 3D shape retrieval

Non-rigid 3D shape retrieval has become an active and important research topic in content-based 3D object retrieval. The aim of this paper is to measure and compare the performance of state-of-the-art methods for non-rigid 3D shape retrieval. The paper develops a new benchmark consisting of 600 non-rigid 3D watertight meshes, which are equally classified into 30 categories, to carry out experiments for 11 different algorithms, whose retrieval accuracies are evaluated using six commonly utilized measures. Models and evaluation tools of the new benchmark are publicly available on our web site [1].

Preface to Special Issue on 3DOR 2011

This Special Issue of The Visual Computer Journal presents four papers that were selected among those presented at the 4th Eurographics Workshop on 3D Object Retrieval, held in Llandudno, UK, on April 04, 2011. All authors of full workshop papers were invited to submit revised and substantially extended versions of their contribution. These underwent a new review process, involving original workshop reviewers, as well as new reviewers. The Eurographics Workshop on 3D Object Retrieval series, founded in 2008, provides a forum for state-of-the-art research in this exciting field at the intersection of Computer Vision, Computer Graphics, Machine Learning, and Human–Computer Interaction. The 3DOR series has gathered momentum and enthusiasm from the research communities addressed, and the 6th Workshop is already planned as a co-event of Eurographics 2013 in Girona, Spain. 3D ob-

Sketch-based shape retrieval

We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an e...

Sketch-based shape retrieval

We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an existing data base of objects. The sketches turn out to be generally quite abstract with large local and global deviations from the original shape. Based on this observation, we decide to use a bag-of-features approach over computer generated line drawings of the objects. We develop a targeted feature transform based on Gabor filters for this system. We can show objectively that this transform is better suited than other approaches from the literature developed for similar tasks. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, our approach is significantly better than any other system described so far.

Non-rigid 3D object retrieval using topological information guided by conformal factors

Combining the properties of conformal geometry and graph-based topological information, a non-rigid 3D object retrieval methodology is proposed, which is both robust and efficient in terms of retrieval accuracy and computation speed. While graph-based methods are robust to non-rigid object deformations, they require intensive computation which can be reduced by the use of appropriate representations, addressed through geometry-based methods. In this respect, we present a 3D object retrieval methodology, which combines the above advantages in a unified manner. Furthermore, we propose a string matching strategy for the comparison of graphs which describe 3D objects.

K-Partite graph reinforcement and its application in multimedia information retrieval

In many example-based information retrieval tasks, example query actually contains multiple sub-queries. For example, in 3D object retrieval, the query is an object described by multiple views. In content-based video retrieval, the query is a video clip that contains multiple frames. Without prior knowledge, the most intuitive approach is to treat the sub-queries equally without difference. In this paper, we propose a k-partite graph reinforcement approach to fuse these sub-queries based on the to-be-retrieved database. The approach first collects the top retrieved results. These results are regarded as pseudo-relevant samples and then a k-partite graph reinforcement is performed on these samples and the query. In the reinforcement process, the weights of the sub-queries are updated by an iterative process. We present experiments on 3D object retrieval and content-based video clip retrieval, and the results demonstrate that our method effectively boosts retrieval performance.

3D model classification based on nonparametric discriminant analysis with kernels

3D model classification has many applications in CAD, 3D object retrieval, and so on. The description of 3D model is crucial but difficult, which leads to the difficulty of classification. The traditional classifier has its limitation in classification of 3D model description. In this paper, we present 3D model classification-based nonparametric discriminant analysis with kernels combined with geometry projection-based histogram model for invariable feature extraction. Firstly, we present nonparametric discriminant analysis with kernels, and secondly, we proposed the invariable feature extraction method with geometry projection-based histogram model. Thirdly, we present the framework of 3D model classification using the proposed nonparametric discriminant analysis with kernels and geometry projection-based histogram model. Finally, we testify the feasibility of the proposed algorithm and performance on 3D model classification. The experimental results show that the proposed scheme is feasible and effective on 3D model classification on the public datasets.

Sparse coding based feature optimisation for robust 3D object retrieval

3D object retrieval is an important research issue in the area of computer graphics for applications such as computer-aided design, simulation, and visualisation. The performance of 3D object retrieval systems is dependent on the development of efficient and effective descriptors and similarity measures. Presented is an effective approach for supporting 3D object retrieval based on optimising the gradient descriptor which has recently been shown to perform among the state of the art. The approach relies on sparse coding and the effectiveness of the method is demonstrated by experimental evaluation.

Active multiple kernel learning for interactive 3D object retrieval systems

An effective relevance feedback solution plays a key role in interactive intelligent 3D object retrieval systems. In this work, we investigate the relevance feedback problem for interactive intelligent 3D object retrieval, with the focus on studying effective machine learning algorithms for improving the user's interaction in the retrieval task. One of the key challenges is to learn appropriate kernel similarity measure between 3D objects through the relevance feedback interaction with users. We address this challenge by presenting a novel framework of Active multiple kernel learning (AMKL), which exploits multiple kernel learning techniques for relevance feedback in interactive 3D object retrieval. The proposed framework aims to efficiently identify an optimal combination of multiple kernels by asking the users to label the most informative 3D images. We evaluate the proposed techniques on a dataset of over 10,000 3D models collected from the World Wide Web. Our experimental results show that the proposed AMKL technique is significantly more effective for 3D object retrieval than the regular relevance feedback techniques widely used in interactive content-based image retrieval, and thus is promising for enhancing user's interaction in such interactive intelligent retrieval systems.

Preface to special issue on 3DOR 2010

This special issue of The Visual Computer encompasses six papers that were selected among those presented at the 3rd Eurographics Workshop on 3D Object Retrieval (May 2, 2010, Norrkoping, Sweden). The authors were invited to submit a revised and substantially extended version of their contribution to The Visual Computer journal which underwent a new review cycle. The Eurographics Workshop on 3D Object Retrieval series, founded in 2008, provides a forum for state-of-the-art research in this exciting research field at the intersection of Computer Vision, Computer Graphics, Machine Learning, and Human–Computer Interaction.

3D object retrieval based on a graph model descriptor

A new 3D object retrieval approach is proposed based on a novel graph model descriptor and a fast graph matching method. Our methodology is made up of two steps. Firstly, a Bayesian network lightfield descriptor (BLD) is built, based on graph model learning, to overcome the disadvantages of the existing view-based methods. The 3D object is put into the lightfield, multi-view images are obtained; then features of the new multi-view images are extracted. Based on the extracted features, a Bayesian network learning algorithm is used to construct the BLD. Secondly, the 3D object is efficiently retrieved, based on graph model matching and learning from relevant feedback. Experimental results demonstrate that our algorithm has better performance and efficiency than the existing view-based methods.