3D Matching Research Articles

A 3D shape matching and retrieval approach based on fusion of curvature and geometric diffusion features

The majority of shape matching and retrieval methods use only one single shape descriptor. Unfortunately, no shape descriptor is sufficient to provide suitable results for all kinds of shapes. The most common way to improve the performance of shape descriptors is to fuse them. In this paper, we propose a new 3D matching and retrieval approach based on a fully unsupervised fusion of curvature and geometric diffusion descriptors. In fact, to improve retrieval precision, we use two descriptors based on local and global features extracted from a shape, and automatically combine these features using a fusion method called Product rule. The Product rule combines values assigned to vertices by the two descriptors. This fusion rule gives better results compared to other well-known fusion schemes such as Max, Min and Linear rules. The proposed approach improves considerably the retrieval precision even with pose changes. This is shown through the retrieval results obtained on several popular 3D shape benchmarks.

A 3D Surface Matching Method Using Keypoint- Based Covariance Matrix Descriptors

A 3D surface is considered one of the most promising tools for representing and recognizing 3D objects. Therefore, 3D surface matching is widely applied to 3D object recognition, retrieval, and so on. In this paper, a 3D surface matching method using a keypoint-based covariance matrix descriptor is proposed, whose purpose is to find correspondences between 3D surfaces (e.g., 3D model and scene) by matching feature points, which are highly repeatable keypoints described by a multi-scale covariance matrix descriptor. A keypoint is detected by analyzing the surface variation index and eigenvalue variation index of a local neighborhood centered at a point. A multi-scale covariance matrix descriptor of a keypoint describes the geometric relation, surface variation gradient, and eigenvalue variation gradient between the keypoint and its neighborhood. The rationale for adopting the keypoint-based covariance matrix descriptor in our proposed 3D surface matching method is that a small number of keypoints with high repeatability can greatly enhance the matching effect of a 3D surface, after being described by a multi-scale covariance matrix descriptor with high descriptiveness. The experimental results also show that our proposed keypoint detection algorithm has higher repeatability than the surface variation index-based and eigenvalue variation index-based detection algorithms; our proposed multi-scale covariance matrix descriptor has higher descriptiveness than spin image, PFH, and 3DSC; our proposed bidirectional nearest-neighbor distance ratio algorithm can obtain better feature matching effect than the nearest-neighbor-based and nearest-neighbor distance-ratio-based feature matching. Finally, our proposed 3D surface matching method has a better matching effect than 3D surface matching methods based on spin image, PFH, and 3DSC on the Stanford 3D Scanning Repository, UWA data set and Bologna data set.

Robust Arbitrary-View Gait Recognition Based on 3D Partial Similarity Matching.

Existing view-invariant gait recognition methods encounter difficulties due to limited number of available gait views and varying conditions during training. This paper proposes gait partial similarity matching that assumes a 3D object shares common view surfaces in significantly different views. Detecting such surfaces aids the extraction of gait features from multiple views; 3D parametric body models are morphed by pose and shape deformation from a template model using 2D gait silhouette as observation. The gait pose is estimated by a level set energy cost function from silhouettes including incomplete ones. Body shape deformation is achieved via Laplacian deformation energy function associated with inpainting gait silhouettes. Partial gait silhouettes in different views are extracted by gait partial region of interest elements selection and re-projected onto 2D space to construct partial gait energy images. A synthetic database with destination views and multi-linear subspace classifier fused with majority voting is used to achieve arbitrary view gait recognition that is robust to varying conditions. Experimental results on CMU, CASIA B, TUM-IITKGP, AVAMVG, and KY4D data sets show the efficacy of the propose method.

A 3D shape matching and retrieval approach based on fusion of curvature and geometric diffusion features

A 3D shape matching and retrieval approach based on fusion of curvature and geometric diffusion features

Applying 3D measurements and computer matching algorithms to two firearm examination proficiency tests

In order for a crime laboratory to assess a firearms examiner’s training, skills, experience, and aptitude, it is necessary for the examiner to participate in proficiency testing. As computer algorithms for comparisons of pattern evidence become more prevalent, it is of interest to test algorithm performance as well, using these same proficiency examinations. This article demonstrates the use of the Congruent Matching Cell (CMC) algorithm to compare 3D topography measurements of breech face impressions and firing pin impressions from a previously distributed firearms proficiency test. In addition, the algorithm is used to analyze the distribution of many comparisons from a collection of cartridge cases used to construct another recent set of proficiency tests. These results are provided along with visualizations that help to relate the features used in optical comparisons by examiners to the features used by computer comparison algorithms.

Sparse 3D directional vertices vs continuous 3D curves: Efficient 3D surface matching and its application for single model face recognition

Traditionally, point clouds and meshes are used to represent and match 3D shapes, which often cannot meet the computational speed and storage space requirements in many 3D data matching and retrieval applications. In this paper, we present a novel 3D directional vertices (3D2V) approach to efficiently represent and match 3D surfaces by much fewer sparsely distributed structured vertices that carry structural information transferred from their deleted neighbouring points. A 3D2V conversion and similarity measurement method is developed to compute the distance between two different 3D2Vs. The performance of the proposed method is evaluated on 3D face recognition using Face Recognition Grand Challenge v2.0 (FRGC v2.0) and GavabDB databases and compared with the curve-based benchmark method. The experimental results demonstrate that the proposed 3D2V method can significantly reduce the data storage requirement and computation time with a moderate increase of accuracy at the same time. It provides a new tool for developing fast 3D surface matching algorithms for large scale 3D data classification and retrieval.

Driving on Point Clouds: Motion Planning, Trajectory Optimization, and Terrain Assessment in Generic Nonplanar Environments

We present a practical approach to global motion planning and terrain assessment for ground robots in generic three‐dimensional (3D) environments, including rough outdoor terrain, multilevel facilities, and more complex geometries. Our method computes optimized six‐dimensional trajectories compliant with curvature and continuity constraints directly on unordered point cloud maps, omitting any kind of explicit surface reconstruction, discretization, or topology extraction. We assess terrain geometry and traversability on demand during motion planning, by fitting robot‐sized planar patches to the map and analyzing the local distribution of map points. Our motion planning approach consists of sampling‐based initial trajectory generation, followed by precise local optimization according to a custom cost measure, using a novel, constraint‐aware trajectory optimization paradigm. We embed these methods in a complete autonomous navigation system based on localization and mapping by means of a 3D laser scanner and iterative closest point matching, suitable for both static and dynamic environments. The performance of the planning and terrain assessment algorithms is evaluated in offline experiments using recorded and simulated sensor data. Finally, we present the results of navigation experiments in three different environments—rough outdoor terrain, a two‐level parking garage, and a dynamic environment, demonstrating how the proposed methods enable autonomous navigation in complex 3D terrain.

Computer-aided detection of pulmonary nodules using dynamic self-adaptive template matching and a FLDA classifier

Improving the performance of computer-aided detection (CAD) system for pulmonary nodules is still an important issue for its future clinical applications. This study aims to develop a new CAD scheme for pulmonary nodule detection based on dynamic self-adaptive template matching and Fisher linear discriminant analysis (FLDA) classifier. We first segment and repair lung volume by using OTSU algorithm and three-dimensional (3D) region growing. Next, the suspicious regions of interest (ROIs) are extracted and filtered by applying 3D dot filtering and thresholding method. Then, pulmonary nodule candidates are roughly detected with 3D dynamic self-adaptive template matching. Finally, we optimally select 11 image features and apply FLDA classifier to reduce false positive detections. The performance of the new method is validated by comparing with other methods through experiments using two groups of public datasets from Lung Image Database Consortium (LIDC) and ANODE09. By a 10-fold cross-validation experiment, the new CAD scheme finally has achieved a sensitivity of 90.24% and a false-positive (FP) of 4.54 FP/scan on average for the former dataset, and a sensitivity of 84.1% with 5.59 FP/scan for the latter. By comparing with other previously reported CAD schemes tested on the same datasets, the study proves that this new scheme can yield higher and more robust results in detecting pulmonary nodules.

Optimizing countershading camouflage

Countershading, the widespread tendency of animals to be darker on the side that receives strongest illumination, has classically been explained as an adaptation for camouflage: obliterating cues to 3D shape and enhancing background matching. However, there have only been two quantitative tests of whether the patterns observed in different species match the optimal shading to obliterate 3D cues, and no tests of whether optimal countershading actually improves concealment or survival. We use a mathematical model of the light field to predict the optimal countershading for concealment that is specific to the light environment and then test this prediction with correspondingly patterned model "caterpillars" exposed to avian predation in the field. We show that the optimal countershading is strongly illumination-dependent. A relatively sharp transition in surface patterning from dark to light is only optimal under direct solar illumination; if there is diffuse illumination from cloudy skies or shade, the pattern provides no advantage over homogeneous background-matching coloration. Conversely, a smoother gradation between dark and light is optimal under cloudy skies or shade. The demonstration of these illumination-dependent effects of different countershading patterns on predation risk strongly supports the comparative evidence showing that the type of countershading varies with light environment.

영상 유도 수술의 환자 및 CT 데이터 좌표계 정렬을 위한 HK 곡률 기술자 기반 표면 정합 방법

In image guided surgery, a patient registration process is a critical process for the successful operation, which is required to use pre-operative images such as CT and MRI during operation. Though several patient registration methods have been studied, we concentrate on one method that utilizes 3D surface measurement data in this paper. First, a hand-held 3D surface measurement device measures the surface of the patient, and secondly this data is matched with CT or MRI data using optimization algorithms. However, generally used ICP algorithm is very slow without a proper initial location and also suffers from local minimum problem. Usually, this problem is solved by manually providing the proper initial location before performing ICP. But, it has a disadvantage that an experience user has to perform the method and also takes a long time. In this paper, we propose a method that can accurately find the proper initial location automatically. The proposed method finds the proper initial location for ICP by converting 3D data to 2D curvature images and performing image matching. Curvature features are robust to the rotation, translation, and even some deformation. Also, the proposed method is faster than traditional methods because it performs 2D image matching instead of 3D point cloud matching.

DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval.

Complex geometric variations of 3D models usually pose great challenges in 3D shape matching and retrieval. In this paper, we propose a novel 3D shape feature learning method to extract high-level shape features that are insensitive to geometric deformations of shapes. Our method uses a discriminative deep auto-encoder to learn deformation-invariant shape features. First, a multiscale shape distribution is computed and used as input to the auto-encoder. We then impose the Fisher discrimination criterion on the neurons in the hidden layer to develop a deep discriminative auto-encoder. Finally, the outputs from the hidden layers of the discriminative auto-encoders at different scales are concatenated to form the shape descriptor. The proposed method is evaluated on four benchmark datasets that contain 3D models with large geometric variations: McGill, SHREC'10 ShapeGoogle, SHREC'14 Human and SHREC'14 Large Scale Comprehensive Retrieval Track Benchmark datasets. Experimental results on the benchmark datasets demonstrate the effectiveness of the proposed method for 3D shape retrieval.

A Survey of 2D and 3D Image Matching in Industrial Object Alignment

Image matching is a practical tool in the computer vision domain. 2D and 3D image matching can be based on (2D) features or on view and point (3D) which can be cloud based. Intensive practical research has been done on both 2D and 3D image matching and the methods are generally designed to meet the requirements of industrial alignment applications and include such aspects as full rotation, scaling and the handling of multiple objects. The aim of this study is the presentation of a review of 2D and 3D image matching and the provision of a comprehensive reference source for those involved in image matching.

Treatment accuracy without rotational setup corrections in intracranial SRT.

The aim of this study was to evaluate the impact of actual rotational setup errors on dose distributions in intracranial stereotactic radiotherapy (SRT) with different alternatives for treatment position selection. A total of 38 SRT fractions from 18 patients were retrospectively evaluated with rotational setup errors obtained from actual treatments. The planning computed tomography (CT) images were rotated according to online cone‐beam CT (CBCT) images and the dose distribution was recalculated to the rotated CT images using three different patient positionings derived from: 1) an automatic 6D match neglecting rotation correction (Auto6D); 2) an automatic 3D match (Auto3D); and 3) a manual 3D match from actual treatment (Treat3D). The mean conformity index (CI) was 0.92 for the original plans and 0.91 for the Auto6D plans. The mean CI decreased significantly (p<0.01) to 0.78 and 0.80 for the Auto3D and the Treat3D plans, respectively. The mean minimum dose of the planning target volume (PTVmin) was 91.9% of the prescribed dose for the original plans and 92.1% for the Auto6D plans, while for the Auto3D and the Treat3D plans PTVmin decreased significantly (p<0.01) to 78.9% and 80.2%, respectively. No significant differences were seen between the Auto6D and the original treatment plans in terms of the dose parameters. However, the Auto3D and the Treat3D plans were statistically significantly inferior (p<0.01) to the Auto6D and the original plans. In addition, a significant negative correlation (p<0.01,|r|>0.38) was found in the Auto3D and the Treat3D cases between the rotation error and CI, PTVmin or minimum dose of gross tumour volume. In SRT, a treatment plan of comparable quality to 6D rotation correction can be achieved by using 6D registration without a rotational correction in the selection of patient positioning. This was demonstrated for typical rotation errors seen in clinical practice.PACS number(s): 87.55, 87.57

DETERMINATION OF STEERING WHEEL ANGLES DURING CAR ALIGNMENT BY IMAGE ANALYSIS METHODS

Abstract. Optical systems for automatic visual inspections are of increasing importance in the field of automation in the industrial domain. A new application is the determination of steering wheel angles during wheel track setting of the final inspection of car manufacturing. The camera has to be positioned outside the car to avoid interruptions of the processes and therefore, oblique images of the steering wheel must be acquired. Three different approaches of computer vision are considered in this paper, i.e. a 2D shape-based matching (by means of a plane to plane rectification of the oblique images and detection of a shape model with a particular rotation), a 3D shape-based matching approach (by means of a series of different perspectives of the spatial shape of the steering wheel derived from a CAD design model) and a point-to-point matching (by means of the extraction of significant elements (e.g. multifunctional buttons) of a steering wheel and a pairwise connection of these points to straight lines). The HALCON system (HALCON, 2016) was used for all software developments and necessary adaptions. As reference a mechanical balance with an accuracy of 0.1° was used. The quality assessment was based on two different approaches, a laboratory test and a test during production process. In the laboratory a standard deviation of ±0.035° (2D shape-based matching), ±0.12° (3D approach) and ±0.029° (point-to-point matching) could be obtained. The field test of 291 measurements (27 cars with varying poses and angles of the steering wheel) results in a detection rate of 100% and ±0.48° (2D matching) and ±0.24° (point-to-point matching). Both methods also fulfil the request of real time processing (three measurements per second).

DETERMINATION OF STEERING WHEEL ANGLES DURING CAR ALIGNMENT BY IMAGE ANALYSIS METHODS

Optical systems for automatic visual inspections are of increasing importance in the field of automation in the industrial domain. A new application is the determination of steering wheel angles during wheel track setting of the final inspection of car manufacturing. The camera has to be positioned outside the car to avoid interruptions of the processes and therefore, oblique images of the steering wheel must be acquired. Three different approaches of computer vision are considered in this paper, i.e. a 2D shape-based matching (by means of a plane to plane rectification of the oblique images and detection of a shape model with a particular rotation), a 3D shape-based matching approach (by means of a series of different perspectives of the spatial shape of the steering wheel derived from a CAD design model) and a point-to-point matching (by means of the extraction of significant elements (e.g. multifunctional buttons) of a steering wheel and a pairwise connection of these points to straight lines). The HALCON system (HALCON, 2016) was used for all software developments and necessary adaptions. As reference a mechanical balance with an accuracy of 0.1° was used. The quality assessment was based on two different approaches, a laboratory test and a test during production process. In the laboratory a standard deviation of ±0.035° (2D shape-based matching), ±0.12° (3D approach) and ±0.029° (point-to-point matching) could be obtained. The field test of 291 measurements (27 cars with varying poses and angles of the steering wheel) results in a detection rate of 100% and ±0.48° (2D matching) and ±0.24° (point-to-point matching). Both methods also fulfil the request of real time processing (three measurements per second).

Stable Marriage with General Preferences

We propose a generalization of the classical stable marriage problem. In our model, the preferences on one side of the partition are given in terms of arbitrary binary relations, which need not be transitive nor acyclic. This generalization is practically well-motivated, and as we show, encompasses the well studied hard variant of stable marriage where preferences are allowed to have ties and to be incomplete. As a result, we prove that deciding the existence of a stable matching in our model is NP-complete. Complementing this negative result we present a polynomial-time algorithm for the above decision problem in a significant class of instances where the preferences are asymmetric. We also present a linear programming formulation whose feasibility fully characterizes the existence of stable matchings in this special case. Finally, we use our model to study a long standing open problem regarding the existence of cyclic 3D stable matchings. In particular, we prove that the problem of deciding whether a fixed 2D perfect matching can be extended to a 3D stable matching is NP-complete, showing this way that a natural attempt to resolve the existence (or not) of 3D stable matchings is bound to fail.

Generic epipolar resampling method for perspective frame camera and linear push-broom sensor

Epipolar rectification aims at resampling stereoscopic images so that conjugate points are located along the same horizontal x-axis. For most stereoscopic digital surface model production algorithms, this rectification is a necessary preprocessing task because it allows reducing the 2D matching to a 1D matching problem. Most epipolar rectification algorithms exist in literature and are mainly for a given type of sensor (frame camera or push-broom sensor). In this paper, we present a generic method for epipolar rectification. This means that the method is applicable for both frame and push-broom cameras. The only difference lies in the projection function used to generate epipolar curves.

3D face recognition using covariance based descriptors

In this paper, we propose a new 3D face recognition method based on covariance descriptors. Unlike feature-based vectors, covariance-based descriptors enable the fusion and the encoding of different types of features and modalities into a compact representation. The covariance descriptors are symmetric positive definite matrices which can be viewed as an inner product on the tangent space of (Symd+) the manifold of Symmetric Positive Definite (SPD) matrices. In this article, we study geodesic distances on the Symd+ manifold and use them as metrics for 3D face matching and recognition. We evaluate the performance of the proposed method on the FRGCv2 and the GAVAB databases and demonstrate its superiority compared to other state of the art methods.

A Mobile Structured Light System for 3D Face Acquisition

A mobile sensor based on fringe projection techniques is developed with the goal of acquiring face 3D and color with a smartphone device. The system consists of a portable pico-projector and an Android-based smartphone. The data acquisition, pattern generation. and reconstruction of the final 3D point cloud are all driven by the smartphone. We present results on the root-mean-square error (RMSE) of the sensor and on 3D face matching.

Decoupled Algorithm for MRI Reconstruction Using Nonlocal Block Matching Model: BM3D-MRI

The block matching 3D (BM3D) is an efficient image model, which has found few applications other than its niche area of denoising. We will develop a magnetic resonance imaging (MRI) reconstruction algorithm, which uses decoupled iterations alternating over a denoising step realized by the BM3D algorithm and a reconstruction step through an optimization formulation. The decoupling of the two steps allows the adoption of a strategy with a varying regularization parameter, which contributes to the reconstruction performance. This new iterative algorithm efficiently harnesses the power of the nonlocal, image-dependent BM3D model. The MRI reconstruction performance of the proposed algorithm is superior to state-of-the-art algorithms from the literature. A convergence analysis of the algorithm is also presented.