Light Field Descriptor Research Articles

A Convolutional Neural Network Approach to the Classification of Engineering Models

This paper presents a deep learning approach for the classification of Engineering (CAD) models using Convolutional Neural Networks (CNNs). Owing to the availability of large annotated datasets and also enough computational power in the form of GPUs, many deep learning-based solutions for object classification have been proposed of late, especially in the domain of images and graphical models. Nevertheless, very few solutions have been proposed for the task of functional classification of CAD models. Hence, for this research, CAD models have been collected from Engineering Shape Benchmark (ESB), National Design Repository (NDR) and augmented with newer models created using a modeling software to form a dataset - `CADNET'. It is proposed to use a residual network architecture for CADNET, inspired by the popular ResNet. A weighted Light Field Descriptor (LFD) scheme is chosen as the method of feature extraction, and the generated images are fed as inputs to the CNN. The problem of class imbalance in the dataset is addressed using a class weights approach. Experiments have been conducted with other signatures such as geodesic distance etc. using deep networks as well as other network architectures on the CADNET. The LFD-based CNN approach using the proposed network architecture, along with gradient boosting yielded the best classification accuracy on CADNET.

A methodology for part classification with supervised machine learning

AbstractIn this paper, we report on a data analysis process for the automated classification of mechanical components. In particular, here, we describe, how to implement a machine learning system for the automated classification of parts belonging to several sub-categories. We collect models that are typically used in the mechanical industry, and then we represent each object by a collection of features. We illustrate how to set-up a supervised multi-layer artificial neural network with an ad-hoc classification schema. We test our solution on a dataset formed by 2354 elements described by 875 features and spanned among 15 sub-categories. We state the accuracy of classification in terms of average area under ROC curves and the ability to classify 606 unknown 3D objects by similarity coefficients. Our parts’ classification system outperforms a classifier based on the Light Field Descriptor, which, as far as we know, actually represents the gold standard for the identification of most types of 3D mechanical objects.

Joint shape and color descriptors for 3D urban model retrieval

ABSTRACTWith the rapid development of photogrammetry, computer vision and three-dimensional (3D) modeling technologies, it is possible to efficiently construct detailed 3D urban models. Accordingly, large corpora of 3D models, such as the Google 3D Warehouse, are now becoming freely available on the web. How to find the proper 3D urban models is a challenging research issue. In this paper, we join shape descriptors and color descriptors for 3D urban model retrieval. The query objects are localized and segmented automatically from the input images by using a new selective search voting algorithm. Through combining the normalization with the light field descriptor, the Horizontal Light Descriptor is introduced to measure the shape similarity among the normalized urban models. The color descriptors are used to represent the color information of the urban models. The two types of descriptors are joined to search 3D urban models similar to the query objects. Experimental results have shown the effectiveness of our approach.

Perception-based shape retrieval for 3D building models

With the help of 3D search engines, a large number of 3D building models can be retrieved freely online. A serious disadvantage of most rotation-insensitive shape descriptors is their inability to distinguish between two 3D building models which are different at their main axes, but appear similar when one of them is rotated. To resolve this problem, we present a novel upright-based normalization method which not only correctly rotates such building models, but also greatly simplifies and accelerates the abstraction and the matching of building models’ shape descriptors. Moreover, the abundance of architectural styles significantly hinders the effective shape retrieval of building models. Our research has shown that buildings with different designs are not well distinguished by the widely recognized shape descriptors for general 3D models. Motivated by this observation and to further improve the shape retrieval quality, a new building matching method is introduced and analyzed based on concepts found in the field of perception theory and the well-known Light Field descriptor. The resulting normalized building models are first classified using the qualitative shape descriptors of Shell and Unevenness which outline integral geometrical and topological information. These models are then put in on orderly fashion with the help of an improved quantitative shape descriptor which we will term as Horizontal Light Field Descriptor, since it assembles detailed shape characteristics. To accurately evaluate the proposed methodology, an enlarged building shape database which extends previous well-known shape benchmarks was implemented as well as a model retrieval system supporting inputs from 2D sketches and 3D models. Various experimental performance evaluation results have shown that, as compared to previous methods, retrievals employing the proposed matching methodology are faster and more consistent with human recognition of spatial objects. In addition these performance evaluation results have verified that the proposed methodology does not sacrifice the matching accuracy while significantly improves the efficiency when matching 3D building models.

3D object retrieval using salient views.

This paper presents a method for selecting salient 2D views to describe 3D objects for the purpose of retrieval. The views are obtained by first identifying salient points via a learning approach that uses shape characteristics of the 3D points (Atmosukarto and Shapiro in International workshop on structural, syntactic, and statistical pattern recognition, 2008; Atmosukarto and Shapiro in ACM multimedia information retrieval, 2008). The salient views are selected by choosing views with multiple salient points on the silhouette of the object. Silhouette-based similarity measures from Chen et al. (Comput Graph Forum 22(3):223-232, 2003) are then used to calculate the similarity between two 3D objects. Retrieval experiments were performed on three datasets: the Heads dataset, the SHREC2008 dataset, and the Princeton dataset. Experimental results show that the retrieval results using the salient views are comparable to the existing light field descriptor method (Chen et al. in Comput Graph Forum 22(3):223-232, 2003), and our method achieves a 15-fold speedup in the feature extraction computation time.

Extended cone-curvature based salient points detection and 3D model retrieval

Local feature extraction of 3D model has become a more and more important aspect in terms of 3D model shape feature extraction. Compared with the global feature, it is more suitable to do the partial retrieval and more robust to the model deformation. In this paper, a local feature called extended cone-curvature feature is proposed to describe the local shape feature of 3D model mesh. Based on the extended cone-curvature feature, salient points and salient regions are extracted by using a new salient point detection method. Then extended cone-curvature feature and local shape distribution feature calculated on the salient regions are used together as shape index, and the earth mover's distance is employed to accomplish similarity measure. After many times' retrieval experiments, the new extended cone-curvature descriptor we propose has more efficient and effective performance than shape distribution descriptor and light field descriptor especially on deformable model retrieval.

3D object retrieval based on a graph model descriptor

A new 3D object retrieval approach is proposed based on a novel graph model descriptor and a fast graph matching method. Our methodology is made up of two steps. Firstly, a Bayesian network lightfield descriptor (BLD) is built, based on graph model learning, to overcome the disadvantages of the existing view-based methods. The 3D object is put into the lightfield, multi-view images are obtained; then features of the new multi-view images are extracted. Based on the extracted features, a Bayesian network learning algorithm is used to construct the BLD. Secondly, the 3D object is efficiently retrieved, based on graph model matching and learning from relevant feedback. Experimental results demonstrate that our algorithm has better performance and efficiency than the existing view-based methods.

AN EFFICIENT 3D MODEL RETRIEVAL BASED ON PRINCIPAL AXES ANALYSIS AND FEATURE INTEGRATION

Three-dimensional (3D) model retrieval has gathered great importance in recent years, since the number of available 3D models on the Internet has drastically increased. Many content-based 3D model retrieval approaches have been proposed. Among these methods, visual similarity-based methods have shown higher retrieval accuracy. However, because these methods capture enormous shape features from different viewpoints or locations, a large amount of calculation and comparison is necessary. Furthermore, there is a trade-off between retrieval accuracy and speed. In this paper, a 3D model retrieval method constituting Continuous Principal Component Analysis (CPCA), Fourier descriptors, and Zernike moments is proposed. CPCA is applied to extract significant shape features based on projecting the model along the principal axes. Then, Fourier descriptors and Zernike moments are used to provide shape descriptors with rotation invariants. In addition, a feature integration process combines them. A strategy of similarity measure is proposed to solve the axes switching problem. To conclude, the experimental results show that the approach outperforms SECTORS2 and D2,18 and has slightly better retrieval results than Light Field Descriptor (LFD)6 and spin-image signatures.3 Moreover, the approach is more efficient and the storage size is much less.

3D Shape description algorithms applied to the problem of model retrieval

AbstractInterest in three-dimensional shape retrieval is currently increasing, driven by two important reasons — the rapid increase of the amount of multimedia data and a noticeable advance in computer hardware and software during recent years. Presently, it is possible to retrieve complicated 3D models in a reasonable span of time thanks to the use of sophisticated 3D shape description algorithms, a feat which was unthinkable a few years ago. The main issue is the efficiency of the approaches, which must work both quickly and reliably. Hence, in this paper four 3D shape description algorithms — Extended Gaussian Image, Shape Distributions, Shape Histograms and Light Field Descriptor — were experimentally compared in order to determine which was most effective. As it turned out, the latter obtained the best retrieval result.