Surface-based Representations Research Articles

Computing the Gromov-Wasserstein Distance between Two Surface Meshes Using Optimal Transport

The Gromov-Wasserstein (GW) formalism can be seen as a generalization of the optimal transport (OT) formalism for comparing two distributions associated with different metric spaces. It is a quadratic optimization problem and solving it usually has computational costs that can rise sharply if the problem size exceeds a few hundred points. Recently fast techniques based on entropy regularization have being developed to solve an approximation of the GW problem quickly. There are issues, however, with the numerical convergence of those regularized approximations to the true GW solution. To circumvent those issues, we introduce a novel strategy to solve the discrete GW problem using methods taken from statistical physics. We build a temperature-dependent free energy function that reflects the GW problem’s constraints. To account for possible differences of scales between the two metric spaces, we introduce a scaling factor s in the definition of the energy. From the extremum of the free energy, we derive a mapping between the two probability measures that are being compared, as well as a distance between those measures. This distance is equal to the GW distance when the temperature goes to zero. The optimal scaling factor itself is obtained by minimizing the free energy with respect to s. We illustrate our approach on the problem of comparing shapes defined by unstructured triangulations of their surfaces. We use several synthetic and “real life” datasets. We demonstrate the accuracy and automaticity of our approach in non-rigid registration of shapes. We provide numerical evidence that there is a strong correlation between the GW distances computed from low-resolution, surface-based representations of proteins and the analogous distances computed from atomistic models of the same proteins.

Subdivide and Conquer: Adapting Non-Manifold Subdivision Surfaces to Surface-Based Representation and Reconstruction of Complex Geological Structures

Methods from the field of computer graphics are the foundation for the representation of geological structures in the form of geological models. However, as many of these methods have been developed for other types of applications, some of the requirements for the representation of geological features may not be considered, and the capacities and limitations of different algorithms are not always evident. In this work, we therefore review surface-based geological modelling methods from both a geological and computer graphics perspective. Specifically, we investigate the use of NURBS (non-uniform rational B-splines) and subdivision surfaces, as two main parametric surface-based modelling methods, and compare the strengths and weaknesses of the two approaches. Although NURBS surfaces have been used in geological modelling, subdivision surfaces as a standard method in the animation and gaming industries have so far received little attention—even if subdivision surfaces support arbitrary topologies and watertight boundary representation, two aspects that make them an appealing choice for complex geological modelling. It is worth mentioning that watertight models are an important basis for subsequent process simulations. Many complex geological structures require a combination of smooth and sharp edges. Investigating subdivision schemes with semi-sharp creases is therefore an important part of this paper, as semi-sharp creases characterise the resistance of a mesh structure to the subdivision procedure. Moreover, non-manifold topologies, as a challenging concept in complex geological and reservoir modelling, are explored, and the subdivision surface method, which is compatible with non-manifold topology, is described. Finally, solving inverse problems by fitting the smooth surfaces to complex geological structures is investigated with a case study. The fitted surfaces are watertight, controllable with control points, and topologically similar to the main geological structure. Also, the fitted model can reduce the cost of modelling and simulation by using a reduced number of vertices in comparison with the complex geological structure.Graphical

Performance scaling for structural MRI surface parcellations: a machine learning analysis in the ABCD Study.

The use of predefined parcellations on surface-based representations of the brain as a method for data reduction is common across neuroimaging studies. In particular, prediction-based studies typically employ parcellation-driven summaries of brain measures as input to predictive algorithms, but the choice of parcellation and its influence on performance is often ignored. Here we employed preprocessed structural magnetic resonance imaging (sMRI) data from the Adolescent Brain Cognitive Development Study® to examine the relationship between 220 parcellations and out-of-sample predictive performance across 45 phenotypic measures in a large sample of 9- to 10-year-old children (N = 9,432). Choice of machine learning (ML) pipeline and use of alternative multiple parcellation-based strategies were also assessed. Relative parcellation performance was dependent on the spatial resolution of the parcellation, with larger number of parcels (up to ~4,000) outperforming coarser parcellations, according to a power-law scaling of between 1/4 and 1/3. Performance was further influenced by the type of parcellation, ML pipeline, and general strategy, with existing literature-based parcellations, a support vector-based pipeline, and ensembling across multiple parcellations, respectively, as the highest performing. These findings highlight the choice of parcellation as an important influence on downstream predictive performance, showing in some cases that switching to a higher resolution parcellation can yield a relatively large boost to performance.

Valproic acid-induced changes of 4D nuclear morphology in astrocyte cells.

Histone deacetylase inhibitors, such as valproic acid (VPA), have important clinical therapeutic and cellular reprogramming applications. They induce chromatin reorganization that is associated with altered cellular morphology. However, there is a lack of comprehensive characterization of VPA-induced changes of nuclear size and shape. Here, we quantify 3D nuclear morphology of primary human astrocyte cells treated with VPA over time (hence, 4D). We compared volumetric and surface-based representations and identified seven features that jointly discriminate between normal and treated cells with 85% accuracy on day 7. From day 3, treated nuclei were more elongated and flattened and then continued to morphologically diverge from controls over time, becoming larger and more irregular. On day 7, most of the size and shape descriptors demonstrated significant differences between treated and untreated cells, including a 24% increase in volume and 6% reduction in extent (shape regularity) for treated nuclei. Overall, we show that 4D morphometry can capture how chromatin reorganization modulates the size and shape of the nucleus over time. These nuclear structural alterations may serve as a biomarker for histone (de-)acetylation events and provide insights into mechanisms of astrocytes-to-neurons reprogramming.

Validation of Group-wise Registration for Surface-based Functional MRI Analysis.

Resting-state functional MRI (rsfMRI) provides important information for studying and mapping the activities and functions of the brain. Conventionally, rsfMRIs are often registered to structural images in the Euclidean space without considering cortical surface geometry. Meanwhile, a surface-based representation offers a relaxed coordinate chart, but this still requires surface registration for group-wise data analysis. In this work, we investigate the performance of two existing surface registration methods in a surface-based rsfMRI analysis framework: FreeSurfer and Hierarchical Spherical Deformation (HSD). To minimize registration bias, we establish shape correspondence using both methods in a group-wise manner that estimates the unbiased average of a given cohort. To evaluate their performance, we focus on neuroanatomical alignment as well as the amount of distortion that can potentially bias surface tessellation for secondary level rsfMRI data analyses. In the pilot analysis, we examine a single timepoint of imaging data from 100 subjects out of an aging cohort. Overall, HSD establishes improved shape correspondence with reduced mean curvature deviation (10.94% less on average per subject, paired t-test: p <10-10) and reduced registration distortion (FreeSurfer: average 41.91% distortion per subject, HSD: 18.63%, paired t-test: p <10-10). Furthermore, HSD introduces less distortion than FreeSurfer in the areas identified in the individual components that were extracted by surface-based independent component analysis (ICA) after spatial smoothing and time series normalization. Consequently, we show that FreeSurfer capture individual components with globally similar but locally different patterns in ICA in visual inspection.

DeepSurf: a surface-based deep learning approach for the prediction of ligand binding sites on proteins.

The knowledge of potentially druggable binding sites on proteins is an important preliminary step toward the discovery of novel drugs. The computational prediction of such areas can be boosted by following the recent major advances in the deep learning field and by exploiting the increasing availability of proper data. In this article, a novel computational method for the prediction of potential binding sites is proposed, called DeepSurf. DeepSurf combines a surface-based representation, where a number of 3D voxelized grids are placed on the protein's surface, with state-of-the-art deep learning architectures. After being trained on the large database of scPDB, DeepSurf demonstrates superior results on three diverse testing datasets, by surpassing all its main deep learning-based competitors, while attaining competitive performance to a set of traditional non-data-driven approaches. The source code of the method along with trained models are freely available at https://github.com/stemylonas/DeepSurf.git. Supplementary data are available at Bioinformatics online.

Virtual reality framework for editing and exploring medial axis representations of nanometric scale neural structures

We present a novel virtual reality (VR) based framework for the exploratory analysis of nanoscale 3D reconstructions of cellular structures acquired from rodent brain samples through serial electron microscopy. The system is specifically targeted on medial axis representations (skeletons) of branched and tubular structures of cellular shapes, and it is designed for providing to domain scientists: i) effective and fast semi-automatic interfaces for tracing skeletons directly on surface-based representations of cells and structures, ii) fast tools for proofreading, i.e., correcting and editing of semi-automatically constructed skeleton representations, and iii) natural methods for interactive exploration, i.e., measuring, comparing, and analyzing geometric features related to cellular structures based on medial axis representations. Neuroscientists currently use the system for performing morphology studies on sparse reconstructions of glial cells and neurons extracted from a sample of the somatosensory cortex of a juvenile rat. The framework runs in a standard PC and has been tested on two different display and interaction setups: PC-tethered stereoscopic head-mounted display (HMD) with 3D controllers and tracking sensors, and a large display wall with a standard gamepad controller. We report on a user study that we carried out for analyzing user performance on different tasks using these two setups.

Evaluation of the standard measurement uncertainty due to the ISO50 surface determination method for dimensional computed tomography

Surface determination is the process by which a CT volume is converted from an image-based representation of an object to a surface-based representation. The process of surface determination relies on well-established image processing algorithms, but these algorithms are not exact, surface determination therefore contributes a component of uncertainty to X-ray CT based dimensional measurements. The purpose of this work is to develop a method for quantifying the standard uncertainty due to surface determination such that this standard uncertainty can be combined with other standard uncertainties in order to calculate the combined standard uncertainty for X-ray CT based dimensional measurements. The proposed method relies on evaluating the mode and the 68% dispersion of the air and material grey values of a given CT data-set. These values are propagated through the ISO50 threshold calculation to give the standard uncertainty of the ISO50 threshold value from which the standard uncertainty due to the ISO50 surface determination method can be evaluated. The proposed method is verified numerically and then demonstrated experimentally for dimensional measurements of an aluminium workpiece, a polymer workpiece, and a steel additively manufactured workpiece. The results show that the measurement uncertainty due to the ISO50 surface determination method is largest for dimensions that have both internal and external surfaces such as wall thicknesses, followed by dimensions such as internal and external diameters, whilst dimensions that are not sensitive to surface determination, for example centre-to-centre distances have the smallest uncertainty due to the ISO50 surface determination method.

Neural volumes

Modeling and rendering of dynamic scenes is challenging, as natural scenes often contain complex phenomena such as thin structures, evolving topology, translucency, scattering, occlusion, and biological motion. Mesh-based reconstruction and tracking often fail in these cases, and other approaches (e.g., light field video) typically rely on constrained viewing conditions, which limit interactivity. We circumvent these difficulties by presenting a learning-based approach to representing dynamic objects inspired by the integral projection model used in tomographic imaging. The approach is supervised directly from 2D images in a multi-view capture setting and does not require explicit reconstruction or tracking of the object. Our method has two primary components: an encoder-decoder network that transforms input images into a 3D volume representation, and a differentiable ray-marching operation that enables end-to-end training. By virtue of its 3D representation, our construction extrapolates better to novel viewpoints compared to screen-space rendering techniques. The encoder-decoder architecture learns a latent representation of a dynamic scene that enables us to produce novel content sequences not seen during training. To overcome memory limitations of voxel-based representations, we learn a dynamic irregular grid structure implemented with a warp field during ray-marching. This structure greatly improves the apparent resolution and reduces grid-like artifacts and jagged motion. Finally, we demonstrate how to incorporate surface-based representations into our volumetric-learning framework for applications where the highest resolution is required, using facial performance capture as a case in point.

Geometric Convolutional Neural Network for Analyzing Surface-Based Neuroimaging Data.

In machine learning, one of the most popular deep learning methods is the convolutional neural network (CNN), which utilizes shared local filters and hierarchical information processing analogous to the brain’s visual system. Despite its popularity in recognizing two-dimensional (2D) images, the conventional CNN is not directly applicable to semi-regular geometric mesh surfaces, on which the cerebral cortex is often represented. In order to apply the CNN to surface-based brain research, we propose a geometric CNN (gCNN) that deals with data representation on a mesh surface and renders pattern recognition in a multi-shell mesh structure. To make it compatible with the conventional CNN toolbox, the gCNN includes data sampling over the surface, and a data reshaping method for the convolution and pooling layers. We evaluated the performance of the gCNN in sex classification using cortical thickness maps of both hemispheres from the Human Connectome Project (HCP). The classification accuracy of the gCNN was significantly higher than those of a support vector machine (SVM) and a 2D CNN for thickness maps generated by a map projection. The gCNN also demonstrated position invariance of local features, which rendered reuse of its pre-trained model for applications other than that for which the model was trained without significant distortion in the final outcome. The superior performance of the gCNN is attributable to CNN properties stemming from its brain-like architecture, and its surface-based representation of cortical information. The gCNN provides much-needed access to surface-based machine learning, which can be used in both scientific investigations and clinical applications.

Measuring urban forms from inter-building distances: Combining MST graphs with a Local Index of Spatial Association

A new method is proposed for characterising local urban patterns at the scale of a large urban region. The approach overcomes the difficulties of surface-based representations of built-up morphologies and provides an efficient way to account for the proximity of built and non-built land. The strength of the approach resides in the direct use of the coordinates of each building, a very parsimonious input of external parameters, and a local spatial statistical perspective. The method consists in, first, constructing and clumping the Minimum Spanning Tree (MST) that links each building to its neighbours in order to obtain connected urban and suburban graph subsets, and second, measuring a Local Index of Spatial Association (LISA) on inter-building distances in order to characterise patterns. The method differentiates heterogeneous settlements from statistically significant homogeneous clusters with either long or short connections between buildings. The method is applied to the Southern periphery of Brussels (Belgium).

Parameterization of White Matter Manifold-Like Structures Using Principal Surfaces

ABSTRACTIn this article, we are concerned with data generated from a diffusion tensor imaging (DTI) experiment. The goal is to parameterize manifold-like white matter tracts, such as the corpus callosum, using principal surfaces. The problem is approached by finding a geometrically motivated surface-based representation of the corpus callosum and visualized fractional anisotropy (FA) values projected onto the surface. The method also applies to any other diffusion summary. An algorithm is proposed that (a) constructs the principal surface of a corpus callosum; (b) flattens the surface into a parametric two-dimensional (2D) map; and (c) projects associated FA values on the map. The algorithm is applied to a longitudinal study containing 466 diffusion tensor images of 176 multiple sclerosis (MS) patients observed at multiple visits. For each subject and visit, the study contains a registered DTI scan of the corpus callosum at roughly 20,000 voxels. Extensive simulation studies demonstrate fast convergence and robust performance of the algorithm under a variety of challenging scenarios.

The role of surface-based representations of shape in visual object recognition.

This study contrasted the role of surfaces and volumetric shape primitives in three-dimensional object recognition. Observers (N = 50) matched subsets of closed contour fragments, surfaces, or volumetric parts to whole novel objects during a whole-part matching task. Three factors were further manipulated: part viewpoint (either same or different between component parts and whole objects), surface occlusion (comparison parts contained either visible surfaces only, or a surface that was fully or partially occluded in the whole object), and target-distractor similarity. Similarity was varied in terms of systematic variation in nonaccidental (NAP) or metric (MP) properties of individual parts. Analysis of sensitivity (d') showed a whole-part matching advantage for surface-based parts and volumes over closed contour fragments--but no benefit for volumetric parts over surfaces. We also found a performance cost in matching volumetric parts to wholes when the volumes showed surfaces that were occluded in the whole object. The same pattern was found for both same and different viewpoints, and regardless of target-distractor similarity. These findings challenge models in which recognition is mediated by volumetric part-based shape representations. Instead, we argue that the results are consistent with a surface-based model of high-level shape representation for recognition.

Reservoir Modeling for Flow Simulation by Use of Surfaces, Adaptive Unstructured Meshes, and an Overlapping-Control-Volume Finite-Element Method

SummaryWe present new approaches to reservoir modeling and flow simulation that dispose of the pillar-grid concept that has persisted since reservoir simulation began. This results in significant improvements to the representation of multiscale geologic heterogeneity and the prediction of flow through that heterogeneity. The research builds on more than 20 years of development of innovative numerical methods in geophysical fluid mechanics, refined and modified to deal with the unique challenges associated with reservoir simulation.Geologic heterogeneities, whether structural, stratigraphic, sedimentologic, or diagenetic in origin, are represented as discrete volumes bounded by surfaces, without reference to a predefined grid. Petrophysical properties are uniform within the geologically defined rock volumes, rather than within grid cells. The resulting model is discretized for flow simulation by use of an unstructured, tetrahedral mesh that honors the architecture of the surfaces. This approach allows heterogeneity over multiple length-scales to be explicitly captured by use of fewer cells than conventional corner-point or unstructured grids.Multiphase flow is simulated by use of a novel mixed finite-element formulation centered on a new family of tetrahedral element types, PN(DG)–PN+1, which has a discontinuous Nth-order polynomial representation for velocity and a continuous (order N +1) representation for pressure. This method exactly represents Darcy-force balances on unstructured meshes and thus accurately calculates pressure, velocity, and saturation fields throughout the domain. Computational costs are reduced through dynamic adaptive-mesh optimization and efficient parallelization. Within each rock volume, the mesh coarsens and refines to capture key flow processes during a simulation, and also preserves the surface-based representation of geologic heterogeneity. Computational effort is thus focused on regions of the model where it is most required.After validating the approach against a set of benchmark problems, we demonstrate its capabilities by use of a number of test models that capture aspects of geologic heterogeneity that are difficult or impossible to simulate conventionally, without introducing unacceptably large numbers of cells or highly nonorthogonal grids with associated numerical errors. Our approach preserves key flow features associated with realistic geologic features that are typically lost. The approach may also be used to capture near-wellbore flow features such as coning, changes in surface geometry across multiple stochastic realizations, and, in future applications, geomechanical models with fracture propagation, opening, and closing.

Cannabis-related episodic memory deficits and hippocampal morphological differences in healthy individuals and schizophrenia subjects.

Cannabis use has been associated with episodic memory (EM) impairments and abnormal hippocampus morphology among both healthy individuals and schizophrenia subjects. Considering the hippocampus' role in EM, research is needed to evaluate the relationship between cannabis-related hippocampal morphology and EM among healthy and clinical groups. We examined differences in hippocampus morphology between control and schizophrenia subjects with and without a past (not current) cannabis use disorder (CUD). Subjects group-matched on demographics included 44 healthy controls (CON), 10 subjects with a CUD history (CON-CUD), 28 schizophrenia subjects with no history of substance use disorders (SCZ), and 15 schizophrenia subjects with a CUD history (SCZ-CUD). Large-deformation, high-dimensional brain mapping with MRI produced surface-based representations of the hippocampus that were compared across all four groups and correlated with EM and CUD history. Surface maps of the hippocampus were generated to visualize morphological differences. CON-CUD and SCZ-CUD were characterized by distinct cannabis-related hippocampal shape differences and parametric deficits in EM performance. Shape differences observed in CON-CUD were associated with poorer EM performance, while shape differences observed in SCZ-CUD were associated with a longer duration of CUD and shorter duration of CUD remission. A past history of CUD may be associated with notable differences in hippocampal morphology and EM impairments among adults with and without schizophrenia. Although the results may be compatible with a causal hypothesis, we must consider that the observed cannabis-related shape differences in the hippocampus could also be explained as biomarkers of a neurobiological susceptibility to poor memory or the effects of cannabis.

A conceptual framework of computations in mid-level vision.

If a picture is worth a thousand words, as an English idiom goes, what should those words—or, rather, descriptors—capture? What format of image representation would be sufficiently rich if we were to reconstruct the essence of images from their descriptors? In this paper, we set out to develop a conceptual framework that would be: (i) biologically plausible in order to provide a better mechanistic understanding of our visual system; (ii) sufficiently robust to apply in practice on realistic images; and (iii) able to tap into underlying structure of our visual world. We bring forward three key ideas. First, we argue that surface-based representations are constructed based on feature inference from the input in the intermediate processing layers of the visual system. Such representations are computed in a largely pre-semantic (prior to categorization) and pre-attentive manner using multiple cues (orientation, color, polarity, variation in orientation, and so on), and explicitly retain configural relations between features. The constructed surfaces may be partially overlapping to compensate for occlusions and are ordered in depth (figure-ground organization). Second, we propose that such intermediate representations could be formed by a hierarchical computation of similarity between features in local image patches and pooling of highly-similar units, and reestimated via recurrent loops according to the task demands. Finally, we suggest to use datasets composed of realistically rendered artificial objects and surfaces in order to better understand a model's behavior and its limitations.

Automatic Alignment of Genus-Zero Surfaces

A new algorithm is presented that provides a constructive way to conformally warp a triangular mesh of genus zero to a destination surface with minimal metric deformation, as well as a means to compute automatically a measure of the geometric difference between two surfaces of genus zero. The algorithm takes as input a pair of surfaces that are topological 2-spheres, each surface given by a distinct triangulation. The algorithm then constructs a map $(f)$ between the two surfaces. First, each of the two triangular meshes is mapped to the unit sphere using a discrete conformal mapping algorithm. The two mappings are then composed with a Möbius transformation to generate the function $(f)$. The Möbius transformation is chosen by minimizing an energy that measures the distance of $(f)$ from an isometry. We illustrate our approach using several "real life" data sets. We show first that the algorithm allows for accurate, automatic, and landmark-free nonrigid registration of brain surfaces. We then validate our approach by comparing shapes of proteins. We provide numerical experiments to demonstrate that the distances computed with our algorithm between low-resolution, surface-based representations of proteins are highly correlated with the corresponding distances computed between high-resolution, atomistic models for the same proteins.

Cannabis-Related Working Memory Deficits and Associated Subcortical Morphological Differences in Healthy Individuals and Schizophrenia Subjects

Cannabis use is associated with working memory (WM) impairments; however, the relationship between cannabis use and WM neural circuitry is unclear. We examined whether a cannabis use disorder (CUD) was associated with differences in brain morphology between control subjects with and without a CUD and between schizophrenia subjects with and without a CUD, and whether these differences related to WM and CUD history. Subjects group-matched on demographics included 44 healthy controls, 10 subjects with a CUD history, 28 schizophrenia subjects with no history of substance use disorders, and 15 schizophrenia subjects with a CUD history. Large-deformation high-dimensional brain mapping with magnetic resonance imaging was used to obtain surface-based representations of the striatum, globus pallidus, and thalamus, compared across groups, and correlated with WM and CUD history. Surface maps were generated to visualize morphological differences. There were significant cannabis-related parametric decreases in WM across groups. Similar cannabis-related shape differences were observed in the striatum, globus pallidus, and thalamus in controls and schizophrenia subjects. Cannabis-related striatal and thalamic shape differences correlated with poorer WM and younger age of CUD onset in both groups. Schizophrenia subjects demonstrated cannabis-related neuroanatomical differences that were consistent and exaggerated compared with cannabis-related differences found in controls. The cross-sectional results suggest that both CUD groups were characterized by WM deficits and subcortical neuroanatomical differences. Future longitudinal studies could help determine whether cannabis use contributes to these observed shape differences or whether they are biomarkers of a vulnerability to the effects of cannabis that predate its misuse.

Recognising Faces: Effects of Lighting Direction, Inversion, and Brightness Reversal

When information about three-dimensional shape obtained from shading and shadows is ambiguous, the visual system favours an interpretation of surface geometry which is consistent with illumination from above. If pictures of top-lit faces are rotated the resulting stimulus is both figurally inverted and illuminated from below. In this study the question of whether the effects of figural inversion and lighting orientation on face recognition are independent or interactive is addressed. Although there was a clear inversion effect for faces illuminated from the front and above, the inversion effect was found to be reduced or eliminated for faces illuminated from below. A strong inversion effect for photographic negatives was also found but in this case the effect was not dependent on the direction of illumination. These findings are interpreted as evidence to suggest that lighting faces from below disrupts the formation of surface-based representations of facial shape.

Spectral feature selection for shape characterization and classification

This paper proposes a framework for selecting the Laplacian eigenvalues of 3D shapes that are more relevant for shape characterization and classification. We demonstrate the redundancy of the information coded by the shape spectrum and discuss the shape characterization capability of the selected eigenvalues. The feature selection methods used to demonstrate our claim are the AdaBoost algorithm and Support Vector Machine. The efficacy of the selection is shown by comparing the results of the selected eigenvalues on shape characterization and classification with those related to the first k eigenvalues, by varying k over the cardinality of the spectrum. Our experiments, which have been performed on 3D objects represented either as triangle meshes or point clouds, show that working directly with point clouds provides classification results that are comparable with respect to those related to surface-based representations. Finally, we discuss the stability of the computation of the Laplacian spectrum to matrix perturbations.