Shape Matching Approach Research Articles

Unsupervised Learning of Robust Spectral Shape Matching

We propose a novel learning-based approach for robust 3D shape matching. Our method builds upon deep functional maps and can be trained in a fully unsupervised manner. Previous deep functional map methods mainly focus on predicting optimised functional maps alone, and then rely on off-the-shelf post-processing to obtain accurate point-wise maps during inference. However, this two-stage procedure for obtaining point-wise maps often yields sub-optimal performance. In contrast, building upon recent insights about the relation between functional maps and point-wise maps, we propose a novel unsupervised loss to couple the functional maps and point-wise maps, and thereby directly obtain point-wise maps without any post-processing. Our approach obtains accurate correspondences not only for near-isometric shapes, but also for more challenging non-isometric shapes and partial shapes, as well as shapes with different discretisation or topological noise. Using a total of nine diverse datasets, we extensively evaluate the performance and demonstrate that our method substantially outperforms previous state-of-the-art methods, even compared to recent supervised methods. Our code is available at https://github.com/dongliangcao/Unsupervised-Learning-of-Robust-Spectral-Shape-Matching.

Non-Isometric Shape Matching via Functional Maps on Landmark-Adapted Bases.

We propose a principled approach for non-isometric landmark-preserving non-rigid shape matching. Our method is based on the functional map framework, but rather than promoting isometries we focus on near-conformal maps that preserve landmarks exactly. We achieve this, first, by introducing a novel landmark-adapted basis using an intrinsic Dirichlet-Steklov eigenproblem. Second, we establish the functional decomposition of conformal maps expressed in this basis. Finally, we formulate a conformally-invariant energy that promotes high-quality landmark-preserving maps, and show how it can be optimized via a variant of the recently proposed ZoomOut method that we extend to our setting. Our method is descriptor-free, efficient and robust to significant mesh variability. We evaluate our approach on a range of benchmark datasets and demonstrate state-of-the-art performance on non-isometric benchmarks and near state-of-the-art performance on isometricones.

Virtual Screening in the Cloud Identifies Potent and Selective ROS1 Kinase Inhibitors.

ROS1 rearrangements account for 1-2% of non-small cell lung cancer patients, yet there are no specifically designed, selective ROS1 therapies in the clinic. Previous knowledge of potent ROS1 inhibitors with selectivity over TrkA, a selected antitarget, enabled virtual screening as a hit finding approach in this project. The ligand-based virtual screening was focused on identifying molecules with a similar 3D shape and pharmacophore to the known actives. To that end, we turned to the AstraZeneca virtual library, estimated to cover 1015 synthesizable make-on-demand molecules. We used cloud computing-enabled FastROCS technology to search the enumerated 1010 subset of the full virtual space. A small number of specific libraries were prioritized based on the compound properties and a medicinal chemistry assessment and further enumerated with available building blocks. Following the docking evaluation to the ROS1 structure, the most promising hits were synthesized and tested, resulting in the identification of several potent and selective series. The best among them gave a nanomolar ROS1 inhibitor with over 1000-fold selectivity over TrkA and, from the preliminary established SAR, these have the potential to be further optimized. Our prospective study describes how conceptually simple shape-matching approaches can identify potent and selective compounds by searching ultralarge virtual libraries, demonstrating the applicability of such workflows and their importance in early drug discovery.

A hybrid topological and shape-matching approach for structure analysis.

Properties of crystalline and amorphous materials are characterized by the underlying long-range and local crystalline order. Deformations and defects are structural hallmarks of plasticity, ice formation, and crystal growth mechanisms. Partitioning topological networks into constituent crystal building blocks, which is the basis of topological identification criteria, is an intuitive approach for classification in both bulk and confinement. However, techniques reliant on the convex hull for assigning orientations of component units fail for non-convex blocks. Here, we propose a new framework, called Topological Unit Matching (TUM), which exploits information from topological criteria for an efficient shape-matching procedure. TUM is a general family of algorithms, capable of quantifying deformations and unambiguously determining grains of bulk and confined ice polymorphs. We show that TUM significantly improves the identification of quasi-one-dimensional ice by including deformed prism blocks. We demonstrate the efficacy of TUM by analyzing supercooled water nanoparticles, amorphous ice, and phase transitions in an ice nanotube. We also illustrate the superiority of TUM in resolving topological defect structures with minimal parameterization.

Elastic Correspondence between Triangle Meshes

AbstractWe propose a novel approach for shape matching between triangular meshes that, in contrast to existing methods, can match crease features. Our approach is based on a hybrid optimization scheme, that solves simultaneously for an elastic deformation of the source and its projection on the target. The elastic energy we minimize is invariant to rigid body motions, and its non‐linear membrane energy component favors locally injective maps. Symmetrizing this model enables feature aligned correspondences even for non‐isometric meshes. We demonstrate the advantage of our approach over state of the art methods on isometric and non‐isometric datasets, where we improve the geodesic distance from the ground truth, the conformal and area distortions, and the mismatch of the mean curvature functions. Finally, we show that our computed maps are applicable for surface interpolation, consistent cross‐field computation, and consistent quadrangular remeshing of a set of shapes.

A novel approach for partial shape matching and similarity based on data envelopment analysis

Due to the growing number of 3D objects in digital libraries, the task of searching and browsing models in an extensive 3D database has been the focus of considerable research in the area. In the last decade, several approaches to retrieve 3D models based on shape similarity have been proposed. The majority of the existing methods addresses the problem of similarity between objects as a global matching problem. Consequently, most of these techniques do not support a part of the object as a query, in addition to their poor performance for classes with globally non-similar shape models and also for articulated objects. The partial matching technique seems to be a suitable solution to these problems. In this paper, we address the problem of shape matching and retrieval. We propose a new approach based on partial matching in which each 3D object is segmented into its constituent parts, and shape descriptors are computed from these elements to compare similarities. Several experiments investigated that our technique enables fast computing for content-based 3D shape retrieval and significantly improves the results of our method based on Data Envelopment Analysis descriptor for global matching.

Left Ventricle Motion Estimation in Cine MRI With Multilayer Iterative Deformable Graph Matching

Quantifying regional myocardial motion and deformation from cardiac magnetic resonance imaging (MRI) plays an important role in clinical applications. In this paper, we present a novel approach for the estimation of left ventricle myocardial motion based on iterative deformable graph matching for cine MRI. Graph matching is a shape matching approach that can be used to determine the correspondence between two objects. However, existing graph matching algorithms are unsuitable for applications with large deformations. In this paper, we propose an iterative deformable graph matching framework for estimating the correspondence between points extracted from left ventricle myocardium at the end-diastolic and end-systolic phases to estimate cardiac motion. A new cost function for graph matching is defined to measure the discrepancy between the nodes and edges of two graphs under a transformation. By introducing a spatial transformation with a sparsity constraint, we can estimate a robust deformation field, alleviating the influence of inevitable graph mismatches. The correspondence between points is then updated by mapping the source graph using the estimated transformation. The cost function is optimized by alternatively optimizing for correspondence and spatial transformation. Furthermore, we propose a multilayer framework to improve correspondence accuracy using a bottom-up matching procedure. This framework estimates the deformation field between an image at the end-systolic phase and an image at the end-diastolic phase in an MRI sequence. Evaluations of two public cardiac datasets indicate that the proposed framework outperforms traditional graph matching algorithms in accuracy and robustness.

Automatic 3D muqarnas architectural patterns reconstruction using plane representation

Abstract The main objective of this paper is to propose an algorithmic way to build the three-dimensional Islamic architectural muqarnas ornament directly from its plane representations (2D Design pattern). Thus, this paper focus on the extraction from the 2D representation of all the structure information needed for the reconstruction of the 3D muqarnas model. For this purpose, the proposed algorithm is based on two main phases. The first one consists to match each 2D-shape with each possible corresponding 3D-model element by using a 2D-to-3D shape matching approach. The second phase concerns the 3D spatial composition which is based on the neighboring properties of each 2D-shape and its identified 3D-element. Another contribution of this paper concerns the construction of a database of the 3D models corresponding to the set of different 2D-shapes which can be found in a 2D plane representation. The proposed algorithm focuses on the reconstruction of the Moroccan-Andalusian muqarnas style. The performance of this algorithm is assessed by reconstructing old muqarnas patterns and also by generating new ones, which preserve its styles’ authenticity.

Determining Probability of Importance of Features in a Sketch

Abstract Sketches can be categorized as personal, shared, persuasive, and handover sketches. Depending on each category, their level of ambiguity also varies. The applications of sketches include conceptual design, eliciting user preferences, shape retrieval, and sketch-based modeling (SBM). There is a need for quantification of uncertainty in sketches in mapping of sketches to three-dimensional (3D) models in sketch-based modeling, in eliciting user preferences, and in tuning the level of uncertainty in sketches at the conceptual design stage. This paper investigates the role of probability of importance in quantifying the level of uncertainty in sketches by raising the following three research questions: How are the features in a sketch ranked? What is the probability of importance of features in a sketch? What is the level of uncertainty in a sketch? This paper presents an improved framework for uncertainty quantification in sketches. The framework is capable of identifying and ranking the features in the sketch, determining their probability of importance, and finally quantifying the level of uncertainty in the sketch. Ranking the features of a sketch is performed by a hierarchical approach, whereas probability of importance is determined by assessing the probability of likeliness using a shape matching approach and a probability transformation. Quantification of uncertainty is accomplished by using the principle of normalization of entropy. A case study of a bicycle sketch is used to demonstrate that the framework eliminates the need of expert input in assessment of uncertainty in sketches and, hence, can be used by design practitioners with limited experience.

Skeleton matching with applications in severe weather detection

Severe weather conditions cause an enormous amount of damages around the globe. Bow echo patterns in radar images are associated with a number of these destructive conditions such as damaging winds, hail, thunderstorms, and tornadoes. They are detected manually by meteorologists. In this paper, we propose an automatic framework to detect these patterns with high accuracy by introducing novel skeletonization and shape matching approaches. In this framework, first we extract regions with high probability of occurring bow echo from radar images and apply our skeletonization method to extract the skeleton of those regions. Next, we prune these skeletons using our innovative pruning scheme with fuzzy logic. Then, using our proposed shape descriptor, Skeleton Context, we can extract bow echo features from these skeletons in order to use them in shape matching algorithm and classification step. The output of classification indicates whether these regions are bow echo with over 97% accuracy.

Spectral Curve Shape Matching Using Derivatives in Hyperspectral Images

Owning to the absorption of radiation by ground materials on specific wavelengths, spectral curve shapes stay relatively stable in hyperspectral images under various conditions and can be used for material identification. Given a reference spectrum, ground materials can be identified by matching the spectral curve shape of the reference spectrum to that of observed spectra. In the past decade, the spectral curve shape matching approach of the Tetracorder system (SCSMT) has been widely used for material identification. However, the SCSMT approach is performed directly on the continuum-removed spectra. It only catches the shape features on a large scale, missing the fine structures on spectral curves. To consider this issue, a new spectral shape matching approach is proposed, which employs the first-order and second-order spectral derivatives to capture the fine structures on spectra. A new metric is designed to balance the contributions of spectral curves, their first-order and second-order derivatives. To evaluate the performance of the proposed approach, it is applied to synthesized spectra as well as to identify oil spills from two practical hyperspectral images. The experimental results show that the proposed approach can improve the detection performance of the SCSMT approach.

A 3D shape matching and retrieval approach based on fusion of curvature and geometric diffusion features

The majority of shape matching and retrieval methods use only one single shape descriptor. Unfortunately, no shape descriptor is sufficient to provide suitable results for all kinds of shapes. The most common way to improve the performance of shape descriptors is to fuse them. In this paper, we propose a new 3D matching and retrieval approach based on a fully unsupervised fusion of curvature and geometric diffusion descriptors. In fact, to improve retrieval precision, we use two descriptors based on local and global features extracted from a shape, and automatically combine these features using a fusion method called Product rule. The Product rule combines values assigned to vertices by the two descriptors. This fusion rule gives better results compared to other well-known fusion schemes such as Max, Min and Linear rules. The proposed approach improves considerably the retrieval precision even with pose changes. This is shown through the retrieval results obtained on several popular 3D shape benchmarks.

A 3D shape matching and retrieval approach based on fusion of curvature and geometric diffusion features

A 3D shape matching and retrieval approach based on fusion of curvature and geometric diffusion features

Measuring Brain Connectivity via Shape Analysis of fMRI Time Courses and Spectra.

We present a shape matching approach for functional magnetic resonance imaging (fMRI) time course and spectral alignment. We use ideas from differential geometry and functional data analysis to define a functional representation for fMRI signals. The space of fMRI functions is then equipped with a reparameterization invariant Riemannian metric that enables elastic alignment of both amplitude and phase of the fMRI time courses as well as their power spectral densities. Experimental results show significant increases in pairwise node to node correlations and coherences following alignment. We apply this method for finding group differences in connectivity between patients with major depression and healthy controls.

Heuristic hybrid genetic algorithm based shape matching approach for the pose detection of backlight units in LCD module assembly

Backlight unit (BLU) and open cell are two critical parts of the liquid crystal display (LCD) module (LCM) which is the most significant component of LCD TV. To place the open cell into a BLU automatically using an assembly robot, this paper proposes a heuristic hybrid genetic algorithm (HHGA) based shape matching approach to detect the pose (orientation and position) of the BLU. The approach takes advantages of line structures of the BLUs to avoid the most time-consuming exhaustive search. Firstly, obtain initial orientation of a BLU by the dominant orientations which can be obtained with the statistical gradient orientation histograms. Secondly, search the optimal pose through the HHGA which mainly consists of local search strategy, crossover strategy, clone strategy, and mutation strategy. Local search strategy is designed according to the rising trend of matching features around the optimal pose and along the lines. Crossover and clone are heuristic strategies designed according to the distribution characteristics of local maxima to produce new meaningful offspring. Mutation is a necessary strategy to keep the diversity of the population. The performance of the proposed approach has been tested on an image database acquired from the LCM assembly lines and compared with standard hybrid GA and exhaustive search by a new statistical indicator. Experimental results show that the proposed approach has a high efficiency within limited time and is suitable for the pose detection of the BLUs.

Deformable 3d shape retrieval using a spectral geometric descriptor

In this paper, we propose a deformable 3D shape matching and retrieval approach using a spectral skeleton that encodes nonrigid object structures. This spectral skeleton is constructed from the second eigenfunction of the Laplace-Beltrami operator defined on the surface of a 3D shape, and thus it is invariant to isometric transformations. In addition to its intrinsic property, our proposed shape descriptor is compact, robust to noise, discriminative, and efficient to compute. We also present a graph matching framework by comparing the shortest paths between skeleton endpoints. Extensive experimental results demonstrate the feasibility of the proposed shape retrieval approach on three standard benchmarks of nonrigid 3D shapes.

Accurate and efficient shape matching approach using vocabularies of multi-feature space representations

Selection of compressed, robust and accurate features is the fundamental ingredient of effective content-based image recognition and retrieval using shape information of objects in the image. In this paper, we present a four-stage system for real-time object recognition and retrieval that employs multiple feature space representation using contour information. In the first stage, we pre-process the shapes to cater for the presence of distortions such as cracks that can significantly distort the contour information of the shape. We then generate multiple feature space representations of shapes to be used later in proposed combination for efficient and accurate retrieval of shapes using a hierarchical indexing structure. To enable real-time image-based shape analysis by enhancing the efficiency and reducing the storage requirement of proposed shape descriptors, we present a quantization approach to generate vocabulary of feature space representation of shapes. These features are then combined in an ensemble for accurate and efficient shape retrieval and recognition in the presence of large shape datasets. The proposed system is evaluated using publicly available shape datasets such as MPEG 7, Swedish leaf and KIMIA 99 datasets. Our approach achieves higher accuracies which are better than state-of-the-art approaches reported in literature whilst looking at a small subset of shapes in dataset.

Stable haptic interaction based on adaptive hierarchical shape matching

In this paper, we present a framework allowing users to interact with geometrically complex 3D deformable objects using (multiple) haptic devices based on an extended shape matching approach. There are two major challenges for haptic-enabled interaction using the shape matching method. The first is how to obtain a rapid deformation propagation when a large number of shape matching clusters exist. The second is how to robustly handle the collision response when the haptic interaction point hits the particle-sampled deformable volume. Our framework extends existing multi-resolution shape matching methods, providing an improved energy convergence rate. This is achieved by using adaptive integration strategies to avoid insignificant shape matching iterations during the simulation. Furthermore, we present a new mechanism called stable constraint particle coupling which ensures consistent deformable behavior during haptic interaction. As demonstrated in our experimental results, the proposed method provides natural and smooth haptic rendering as well as efficient yet stable deformable simulation of complex models in real time.

Physics-based shape matching for intraoperative image guidance.

Soft-tissue deformations can severely degrade the validity of preoperative planning data during computer assisted interventions. Intraoperative imaging such as stereo endoscopic, time-of-flight or, laser range scanner data can be used to compensate these movements. In this context, the intraoperative surface has to be matched to the preoperative model. The shape matching is especially challenging in the intraoperative setting due to noisy sensor data, only partially visible surfaces, ambiguous shape descriptors, and real-time requirements. A novel physics-based shape matching (PBSM) approach to register intraoperatively acquired surface meshes to preoperative planning data is proposed. The key idea of the method is to describe the nonrigid registration process as an electrostatic-elastic problem, where an elastic body (preoperative model) that is electrically charged slides into an oppositely charged rigid shape (intraoperative surface). It is shown that the corresponding energy functional can be efficiently solved using the finite element (FE) method. It is also demonstrated how PBSM can be combined with rigid registration schemes for robust nonrigid registration of arbitrarily aligned surfaces. Furthermore, it is shown how the approach can be combined with landmark based methods and outline its application to image guidance in laparoscopic interventions. A profound analysis of the PBSM scheme based on in silico and phantom data is presented. Simulation studies on several liver models show that the approach is robust to the initial rigid registration and to parameter variations. The studies also reveal that the method achieves submillimeter registration accuracy (mean error between 0.32 and 0.46 mm). An unoptimized, single core implementation of the approach achieves near real-time performance (2 TPS, 7-19 s total registration time). It outperforms established methods in terms of speed and accuracy. Furthermore, it is shown that the method is able to accurately match partial surfaces. Finally, a phantom experiment demonstrates how the method can be combined with stereo endoscopic imaging to provide nonrigid registration during laparoscopic interventions. The PBSM approach for surface matching is fast, robust, and accurate. As the technique is based on a preoperative volumetric FE model, it naturally recovers the position of volumetric structures (e.g., tumors and vessels). It cannot only be used to recover soft-tissue deformations from intraoperative surface models but can also be combined with landmark data from volumetric imaging. In addition to applications in laparoscopic surgery, the method might prove useful in other areas that require soft-tissue registration from sparse intraoperative sensor data (e.g., radiation therapy).

An Enhanced Graph Representation and Heuristic Tabu Search Approach for Flexible and Efficient 3D Shape Matching

3D shape matching is widely used in engineering design for model retrieval, shape similarity assessment or collaborative development. In this paper, an enhanced graph representation and heuristic tabu search based approach is presented to enable flexible and efficient 3D shape matching. The core idea includes (1) generic shape features are recognized from boundary representation (B-rep) of 3D shape as many as possible to incorporate domain design knowledge, (2) an enhanced graph representation of 3D shape is constructed by mixing faces of B-rep and recognized features, to achieve meaningful matching results at low-level of faces or high-level of features satisfying various design intents, and (3) a tabu list of possible improper matches is built to reduce the search space so that the optimal result can be efficiently obtained. Finally, Two examples are demonstrated to show that both two levels of 3D shape matching results can be efficiently obtained for various design intents in the engineering applications, only not more than 18% computation time is required when compared with a typical shape matching method, and it takes only 20 s when the number of matching nodes is more than 460.