Shape Embedding Research Articles

A Hybrid Recognition Framework for Highly Interacting Machining Features Based on Primitive Decomposition, Learning and Reconstruction

For the highly interacting machining features, Layered Projection Decomposition Method presents inferior recognition efficiency and accuracy, due to its high-cost 3D projection and failures in determining projection faces for internal occluded faces. To address these issues, we propose a potential hybrid recognition framework. We first introduce a straightforward adjacent projection wire (APW) over UV wires, automatically restoring the full projection wires from highly interacting features. Building on APWs, an efficient hybrid boundary representation and its corresponding unambiguous primitive definitions are proposed by combing with graph-based boundary representations. Subsequently, we design an efficient primitive decomposition method by introducing primitive boundary matching to decide the initial projection faces, and introducing iterative projection boundary expansion to complete the full primitives from occluded faces. Moreover, we establish an efficient Graph Neural Network to learn the distinguishable distributions over the decomposed primitives. Specifically, an Adjacency Attention Unit is proposed to automatically perceive the influence weight of adjacent nodes, leading to more discriminative self-adaptive shape embedding for efficient primitive recognition. Finally, we summarize convenient reconstruction rules to correct the wrong predictions of feature faces with indistinguishable adjacent relationships. To evaluate the effectiveness of the proposed recognition framework, CAD models of complex aircraft structural parts are collected to present a challenging machining feature dataset. Extensive numerical experiments demonstrate that the proposed hybrid recognition framework enables significant improvements over the state-of-the-art machining feature recognition techniques.

Shape meets Euclid: Integrating shape computation with ruler and compass procedures

The recent success of a class of mechanical applications of shape rules in CAD shape grammar interpreters has produced a renewed interest in the design and support of shape embedding and shape rewrite technologies. Despite this, some fundamental constructions in rule-based modeling, namely, any of Euclid's straightedge and compass constructions, appear awkward to simulate within the fuse-embed cycle of a shape grammar interpreter. It is suggested here that this problem of modeling Euclid's straightedge and compass constructions can be efficiently addressed with the introduction of a new schema rule leveraging symbolic representations of shape. Two applications based on the usage of three instances of this schema rule are given to show the simulation of straightedge and compass constructions in a general purpose 2D CAD shape grammar interpreter, the Shape Machine for Rhino. A reflection upon the usage of rule schemas in shape grammar interpreters is given in the end.

Shape embedding and retrieval in multi-flow deformation

We propose a unified 3D flow framework for joint learning of shape embedding and deformation for different categories. Our goal is to recover shapes from imperfect point clouds by fitting the best shape template in a shape repository after deformation. Accordingly, we learn a shape embedding for template retrieval and a flow-based network for robust deformation. We note that the deformation flow can be quite different for different shape categories. Therefore, we introduce a novel multi-hub module to learn multiple modes of deformation to incorporate such variation, providing a network which can handle a wide range of objects from different categories. The shape embedding is designed to retrieve the best-fit template as the nearest neighbor in a latent space. We replace the standard fully connected layer with a tiny structure in the embedding that significantly reduces network complexity and further improves deformation quality. Experiments show the superiority of our method to existing state-of-the-art methods via qualitative and quantitative comparisons. Finally, our method provides efficient and flexible deformation that can further be used for novel shape design.

Learning Canonical Embeddings for Unsupervised Shape Correspondence With Locally Linear Transformations.

We present a new approach to unsupervised shape correspondence learning between pairs of point clouds. We make the first attempt to adapt the classical locally linear embedding algorithm (LLE)-originally designed for nonlinear dimensionality reduction-for shape correspondence. The key idea is to find dense correspondences between shapes by first obtaining high-dimensional neighborhood-preserving embeddings of low-dimensional point clouds and subsequently aligning the source and target embeddings using locally linear transformations. We demonstrate that learning the embedding using a new LLE-inspired point cloud reconstruction objective results in accurate shape correspondences. More specifically, the approach comprises an end-to-end learnable framework of extracting high-dimensional neighborhood-preserving embeddings, estimating locally linear transformations in the embedding space, and reconstructing shapes via divergence measure-based alignment of probability density functions built over reconstructed and target shapes. Our approach enforces embeddings of shapes in correspondence to lie in the same universal/canonical embedding space, which eventually helps regularize the learning process and leads to a simple nearest neighbors approach between shape embeddings for finding reliable correspondences. Comprehensive experiments show that the new method makes noticeable improvements over state-of-the-art approaches on standard shape correspondence benchmark datasets covering both human and nonhuman shapes.

Recognition of building shape in maps using deep graph filter neural network

Shape is one of the core features of the buildings which are the main elements of the map. The building shape recognition is widely used in many spatial applications. Due to the irregularity of the building contour, it is still challenging for building shape recognition. Inspired by graph signal processing theory, we propose a deep graph filter neural network (DGFN) for the shape recognition of buildings in maps. First, we regard shape recognition as a combination of subjective and objective graph signal filtering process. Second, we construct a shape features extraction framework from the perspective of shape details, shape structure and shape local information. Third, DGFN model can fulfil the tasks of shape classification and shape embedding of building at the same time. Finally, multi angle experiments verify our viewpoint of shape recognition mechanism, and the comparison with similar algorithms proves the high accuracy and availability of DGFN model.

Automatic Image Guidance for Assessment of Placenta Location in Ultrasound Video Sweeps.

Ultrasound-based assistive tools are aimed at reducing the high skill needed to interpret a scan by providing automatic image guidance. This may encourage uptake of ultrasound (US) clinical assessments in rural settings in low- and middle-income countries (LMICs), where well-trained sonographers can be scarce. This paper describes a new method that automatically generates an assistive video overlay to provide image guidance to a user to assess placenta location. The user captures US video by following a sweep protocol that scans a U-shape on the lower maternal abdomen. The sweep trajectory is simple and easy to learn. We initially explore a 2-D embedding of placenta shapes, mapping manually segmented placentas in US video frames to a 2-D space. We map 2013 frames from 11 videos. This provides insight into the spectrum of placenta shapes that appear when using the sweep protocol. We propose classification of the placenta shapes from three observed clusters: complex, tip and rectangular. We use this insight to design an effective automatic segmentation algorithm, combining a U-Net with a CRF-RNN module to enhance segmentation performance with respect to placenta shape. The U-Net+CRF-RNN algorithm automatically segments the placenta and maternal bladder. We assess segmentation performance using both area and shape metrics. We report results comparable to the state-of-the-art for automatic placenta segmentation on the Dice metric, achieving 0.83 ± 0.15 evaluated on 2127 frames from 10 videos. We also qualitatively evaluate 78,308 frames from 135 videos, assessing if the anatomical outline is correctly segmented. We found that addition of the CRF-RNN improves over a baseline U-Net when faced with a complex placenta shape, which we observe in our 2-D embedding, up to 14% with respect to the percentage shape error. From the segmentations, an assistive video overlay is automatically constructed that (i) highlights the placenta and bladder, (ii) determines the lower placenta edge and highlights this location as a point and (iii) labels a 2-cm clearance on the lower placenta edge. The 2-cm clearance is chosen to satisfy current clinical guidelines. We propose to assess the placenta location by comparing the 2-cm region and the bottom of the bladder, which represents a coarse localization of the cervix. Anatomically, the bladder must sit above the cervix region. We present proof-of-concept results for the video overlay.

Implementation of shape embedding in 2D CAD systems

The process of shape embedding, that is, the inquiry whether for two shapes u and w there is a transformation f that embeds the shape f(u) in w is the most critical – and elusive – process for all shape grammar interpreters implemented within CAD systems. This paper identifies three major challenges underlying the implementation of shape embedding within a CAD environment and proposes three corresponding solutions. The focus is given here in the calculation of shape embedding for shapes made up of lines in a 2D CAD environment to provide the testbed for the mechanisms proposed. The integration of these three sets of algorithms into one coherent framework produces the blueprint of Shape Machine, a general-purpose shape grammar interpreter implemented in Rhinoceros, a NURBS 2D/3D CAD software. Some preliminary thoughts regarding the impact of the proposed shape-rewrite modeler in CAD industry and more broadly, in design education are given in the end.

Shape Embedding: A Means of Superimposing Alternative Design Descriptions on Shape Models

In 1989, Shapiro and Voelcker introduced three principles for relating shape and function with the goal of establishing an appropriate formal role for geometry in a theory of mechanical design. This technical note contributes to the establishment of such a theory by using an example from their technical note to illustrate the potential value of incorporating shape embedding, used in the shape computation and shape grammar community to support shape synthesis, into CAD packages that are used primarily to support shape definition and analysis. • Recollections of collaborations with Prof Herb Voelcker since the early 1970s. • There is a need, in engineering design, for multiple descriptions of a given design. • These descriptions are used for different purposes through the lifecycle of a product. • Embedding is a mechanism that can be used to superimpose multiple design descriptions on a given shape model. • The use of embedding promises underlying consistency across design descriptions without the need for deep integration of said descriptions.

Echocardiography Segmentation With Enforced Temporal Consistency.

Convolutional neural networks (CNN) have demonstrated their ability to segment 2D cardiac ultrasound images. However, despite recent successes according to which the intra-observer variability on end-diastole and end-systole images has been reached, CNNs still struggle to leverage temporal information to provide accurate and temporally consistent segmentation maps across the whole cycle. Such consistency is required to accurately describe the cardiac function, a necessary step in diagnosing many cardiovascular diseases. In this paper, we propose a framework to learn the 2D+time apical long-axis cardiac shape such that the segmented sequences can benefit from temporal and anatomical consistency constraints. Our method is a post-processing that takes as input segmented echocardiographic sequences produced by any state-of-the-art method and processes it in two steps to (i) identify spatio-temporal inconsistencies according to the overall dynamics of the cardiac sequence and (ii) correct the inconsistencies. The identification and correction of cardiac inconsistencies relies on a constrained autoencoder trained to learn a physiologically interpretable embedding of cardiac shapes, where we can both detect and fix anomalies. We tested our framework on 98 full-cycle sequences from the CAMUS dataset, which are available alongside this paper. Our temporal regularization method not only improves the accuracy of the segmentation across the whole sequences, but also enforces temporal and anatomical consistency.

GeoRec: Geometry-enhanced semantic 3D reconstruction of RGB-D indoor scenes

Semantic indoor 3D modeling with multi-task deep neural networks is an efficient and low-cost way for reconstructing an indoor scene with geometrically complete room structure and semantic 3D individuals. Challenged by the complexity and clutter of indoor scenarios, the semantic reconstruction quality of current methods is still limited by the insufficient exploration and learning of 3D geometry information. To this end, this paper proposes an end-to-end multi-task neural network for geometry-enhanced semantic 3D reconstruction of RGB-D indoor scenes (termed as GeoRec). In the proposed GeoRec, we build a geometry extractor that can effectively learn geometry-enhanced feature representation from depth data, to improve the estimation accuracy of layout, camera pose and 3D object bounding boxes. We also introduce a novel object mesh generator that strengthens the reconstruction robustness of GeoRec to indoor occlusion with geometry-enhanced implicit shape embedding. With the parsed scene semantics and geometries, the proposed GeoRec reconstructs an indoor scene by placing reconstructed object mesh models with 3D object detection results in the estimated layout cuboid. Extensive experiments conducted on two benchmark datasets show that the proposed GeoRec yields outstanding performance with 5.19×10-3 mean chamfer distance error for object reconstruction on the challenging Pix3D dataset, 70.45% mAP for 3D object detection and 77.1% 3D mIoU for layout estimation on the commonly-used SUN RGB-D dataset. Especially, the mesh reconstruction sub-network of GeoRec trained on Pix3D can be directly transferred to SUN RGB-D without any fine-tuning, manifesting a high generalization ability.

What shape grammars do that CAD should: the 14 cases of shape embedding

Abstract Shape queries based on shape embedding under a given Euclidean, affine, or linear transformation are absent from current CAD systems. The only systems that have attempted to implement shape embedding are the shape grammar interpreters albeit with promising but inconclusive results. The work here identifies all possible 14 cases of shape embedding with respect to the number of available registration points, four for determinate cases and ten for indeterminate ones, and an approach is sketched to take on the complexities underlying the indeterminate cases. All visual calculations are done with shapes consisting of straight lines in the Euclidean plane within the algebra U ij for i = 1 the dimension of lines and j = 2 the dimension of space in which the lines are defined, transformed and combined. Aspects of interface design and integration to current work design workflows are deliberately left aside.

Dress With Style: Learning Style From Joint Deep Embedding of Clothing Styles and Body Shapes

Body shape is about proportion, and fashion style is all about dressing those proportions to look their very best. Figuring out the styles to suit a body shape can be a daunting task for many people. It is, therefore, essential to develop a framework for learning the compatibility of body shapes and clothing styles. Though fashion designers and fashion stylists have analyzed the correlation between human body shapes and fashion styles for a long time, this issue did not receive much attention in multimedia science. In this paper, we present a novel style recommender, on the basis of the user's body attributes. The rich amount of fashion styling knowledge from social big data is exploited for this purpose. We first construct a joint embedding of clothing styles and human body measurements with deep multimodal representation learning on a reference dataset that has been sorted to meet the fashion rules. We then discover the relevant semantic features by propagation and selection in clothing style and body shape graphs. Experiments demonstrate the effectiveness of the proposed framework when compared with several baseline methods.

Learning Discriminative and Generative Shape Embeddings for Three-Dimensional Shape Retrieval

As an important solution for 3D shape retrieval, a multi-view shape descriptor has achieved impressive performance. One crucial part of view-based shape descriptors is to interpret 3D structures through various 2D observations. Most existing methods like MVCNN believe that a strong classification model trained with deep learning, can often provide an efficient shape embedding for 3D shape retrieval. However, these methods pay much attention to discriminative models and none of them necessarily incorporate the underlying 3D properties of the objects from 2D images. In this paper, we present a novel encoder-decoder recurrent feature aggregation network (ERFA-Net) to address this problem. Aiming at emphasizing the 3D properties of 3D shapes in the fusion of multiple view features, 3D properties prediction tasks are introduced into the 3D shape retrieval. Specifically, an image sequence of the shape is recurrently aggregated into a discriminative shape embedding based on LSTM network, and then this latent shape embedding is trained to predict the original voxel grids and estimate images of unseen viewpoints. This generation task gives an effective supervision which makes the network exploit 3D properties of shapes through various 2D images. Our method achieves the state-of-the-art performance for 3D shape retrieval, on two large-scale 3D shape datasets, ModelNet and ShapeNetCore55. Extensive experiments show that the proposed 3D representation performs robust discrimination against view occlusion, and strong generation ability for various 3D shape tasks.

Angular Triplet-Center Loss for Multi-View 3D Shape Retrieval

How to obtain the desirable representation of a 3D shape, which is discriminative across categories and polymerized within classes, is a significant challenge in 3D shape retrieval. Most existing 3D shape retrieval methods focus on capturing strong discriminative shape representation with softmax loss for the classification task, while the shape feature learning with metric loss is neglected for 3D shape retrieval. In this paper, we address this problem based on the intuition that the cosine distance of shape embeddings should be close enough within the same class and far away across categories. Since most of 3D shape retrieval tasks use cosine distance of shape features for measuring shape similarity, we propose a novel metric loss named angular triplet-center loss, which directly optimizes the cosine distances between the features. It inherits the triplet-center loss property to achieve larger inter-class distance and smaller intra-class distance simultaneously. Unlike previous metric loss utilized in 3D shape retrieval methods, where Euclidean distance is adopted and the margin design is difficult, the proposed method is more convenient to train feature embeddings and more suitable for 3D shape retrieval. Moreover, the angle margin is adopted to replace the cosine margin in order to provide more explicit discriminative constraints on an embedding space. Extensive experimental results on two popular 3D object retrieval benchmarks, ModelNet40 and ShapeNetCore 55, demonstrate the effectiveness of our proposed loss, and our method has achieved state-ofthe-art results on various 3D shape datasets.

Learning to Predict Ischemic Stroke Growth on Acute CT Perfusion Data by Interpolating Low-Dimensional Shape Representations.

Cerebrovascular diseases, in particular ischemic stroke, are one of the leading global causes of death in developed countries. Perfusion CT and/or MRI are ideal imaging modalities for characterizing affected ischemic tissue in the hyper-acute phase. If infarct growth over time could be predicted accurately from functional acute imaging protocols together with advanced machine-learning based image analysis, the expected benefits of treatment options could be better weighted against potential risks. The quality of the outcome prediction by convolutional neural networks (CNNs) is so far limited, which indicates that even highly complex deep learning algorithms are not fully capable of directly learning physiological principles of tissue salvation through weak supervision due to a lack of data (e.g., follow-up segmentation). In this work, we address these current shortcomings by explicitly taking into account clinical expert knowledge in the form of segmentations of the core and its surrounding penumbra in acute CT perfusion images (CTP), that are trained to be represented in a low-dimensional non-linear shape space. Employing a multi-scale CNN (U-Net) together with a convolutional auto-encoder, we predict lesion tissue probabilities for new patients. The predictions are physiologically constrained to a shape embedding that encodes a continuous progression between the core and penumbra extents. The comparison to a simple interpolation in the original voxel space and an unconstrained CNN shows that the use of such a shape space can be advantageous to predict time-dependent growth of stroke lesions on acute perfusion data, yielding a Dice score overlap of 0.46 for predictions from expert segmentations of core and penumbra. Our interpolation method models monotone infarct growth robustly on a linear time scale to automatically predict clinically plausible tissue outcomes that may serve as a basis for more clinical measures such as the expected lesion volume increase and can support the decision making on treatment options and triage.

Computational anatomy and diffeomorphometry: A dynamical systems model of neuroanatomy in the soft condensed matter continuum.

The nonlinear systems models of computational anatomy that have emerged over the past several decades are a synthesis of three significant areas of computational science and biological modeling. First is the algebraic model of biological shape as a Riemannian orbit, a set of objects under diffeomorphic action. Second is the embedding of anatomical shapes into the soft condensed matter physics continuum via the extension of the Euler equations to geodesic, smooth flows with inverses, encoding divergence for the compressibility of atrophy and expansion of growth. Third, is making human shape and form a metrizable space via geodesic connections of coordinate systems. These three themes place our formalism into the modern data science world of personalized medicine supporting inference of high-dimensional anatomical phenotypes for studying neurodegeneration and neurodevelopment. The dynamical systems model of growth and atrophy that emerges is one which is organized in terms of forces, accelerations, velocities, and displacements, with the associated Hamiltonian momentum and the diffeomorphic flow acting as the state, and the smooth vector field the control. The forces that enter the model derive from external measurements through which the dynamical system must flow, and the internal potential energies of structures making up the soft condensed matter. We examine numerous examples on growth and atrophy. This article is categorized under: Analytical and Computational Methods > Computational Methods Laboratory Methods and Technologies > Imaging Models of Systems Properties and Processes > Organ, Tissue, and Physiological Models.

Emphasizing 3D Properties in Recurrent Multi-View Aggregation for 3D Shape Retrieval

Multi-view based shape descriptors have achieved impressive performance for 3D shape retrieval. The core of view-based methods is to interpret 3D structures through 2D observations. However, most existing methods pay more attention to discriminative models and none of them necessarily incorporate the 3D properties of the objects. To resolve this problem, we propose an encoder-decoder recurrent feature aggregation network (ERFA-Net) to emphasize the 3D properties of 3D shapes in multi-view features aggregation. In our network, a view sequence of the shape is trained to encode a discriminative shape embedding and estimate unseen rendered views of any viewpoints. This generation task gives an effective supervision which makes the network exploit 3D properties of shapes through various 2D images. During feature aggregation, a discriminative feature representation across multiple views is effectively exploited based on LSTM network. The proposed 3D representation has following advantages against other state-of-the-art: 1) it performs robust discrimination under the existence of noise such as view missing and occlusion, because of the improvement brought by 3D properties. 2) it has strong generative capabilities, which is useful for various 3D shape tasks. We evaluate ERFA-Net on two popular 3D shape datasets, ModelNet and ShapeNetCore55, and ERFA-Net outperforms the state-of-the-art methods significantly. Extensive experiments show the effectiveness and robustness of the proposed 3D representation.

From Images to 3D Shape Attributes.

Our goal in this paper is to investigate properties of 3D shape that can be determined from a single image. We define 3D shape attributes-generic properties of the shape that capture curvature, contact and occupied space. Our first objective is to infer these 3D shape attributes from a single image. A second objective is to infer a 3D shape embedding-a low dimensional vector representing the 3D shape. We study how the 3D shape attributes and embedding can be obtained from a single image by training a Convolutional Neural Network (CNN) for this task. We start with synthetic images so that the contribution of various cues and nuisance parameters can be controlled. We then turn to real images and introduce a large scale image dataset of sculptures containing 143K images covering 2197 works from 242 artists. For the CNN trained on the sculpture dataset we show the following: (i) which regions of the imaged sculpture are used by the CNN to infer the 3D shape attributes; (ii) that the shape embedding can be used to match previously unseen sculptures largely independent of viewpoint; and (iii) that the 3D attributes generalize to images of other (non-sculpture) object classes.

Learning part-in-whole relation of 3D shapes for part-based 3D model retrieval

Given a query that specifies partial 3D shape, a Part-based 3D Model Retrieval (P3DMR) system finds 3D shapes whose part or parts matches the query. An approach to P3DMR is to partition or segment whole models into sub-parts and performs query-part-to-target-parts matching. Whatever the definition of part, e.g., a rectangular volume in Euclidean space or a part segmented on a mesh manifold, the computation will be very costly. The part-whole matching must account for, for each 3D whole shape in a database, varying position, scale and orientation of the segmented sub parts. Another approach, in an attempt to make part-whole matching efficient, tries to approximate part-whole inclusion test with a single comparison between a pair of features, one representing the part-based query and the other representing the whole shape. Aggregation of local geometrical features of parts into a feature per whole 3D shape, e.g., via Bag-of-Features approach, is an example. This approach so far suffered from inaccuracy as the aggregation is not optimized for part-whole inclusion test of 3D shapes. This paper proposes a novel P3DMR algorithm called Part-Whole Relation Embedding network (PWRE-net) that effectively and efficiently performs part-whole inclusion test via learned embedding into a common feature space. Using deep neural network, the PWRE-net learns, from a large number of part-whole shape pairs, a common embedding of partial shapes and their associated whole shapes. For the training, training datasets containing part-whole shape pairs are created automatically from unlabeled 3D models. Experimental evaluation shows that PWRE-net outperforms existing algorithms both in terms of retrieval accuracy and efficiency.

Shape‐from‐Operator: Recovering Shapes from Intrinsic Operators

AbstractWe formulate the problem of shape‐from‐operator (SfO), recovering an embedding of a mesh from intrinsic operators defined through the discrete metric (edge lengths). Particularly interesting instances of our SfO problem include: shape‐from‐Laplacian, allowing to transfer style between shapes; shape‐from‐difference operator, used to synthesize shape analogies; and shape‐from‐eigenvectors, allowing to generate ‘intrinsic averages’ of shape collections. Numerically, we approach the SfO problem by splitting it into two optimization sub‐problems: metric‐from‐operator (reconstruction of the discrete metric from the intrinsic operator) and embedding‐from‐metric (finding a shape embedding that would realize a given metric, a setting of the multidimensional scaling problem). We study numerical properties of our problem, exemplify it on several applications, and discuss its imitations.