Cortical Surface Registration Research Articles

Longitudinally consistent registration and parcellation of cortical surfaces using semi-supervised learning

Temporally consistent and accurate registration and parcellation of longitudinal cortical surfaces is of great importance in studying longitudinal morphological and functional changes of human brains. However, most existing methods are developed for registration or parcellation of a single cortical surface. When applying to longitudinal studies, these methods independently register/parcellate each surface from longitudinal scans, thus often generating longitudinally inconsistent and inaccurate results, especially in small or ambiguous cortical regions. Essentially, longitudinal cortical surface registration and parcellation are highly correlated tasks with inherently shared constraints on both spatial and temporal feature representations, which are unfortunately ignored in existing methods. To this end, we unprecedentedly propose a novel semi-supervised learning framework to exploit these inherent relationships from limited labeled data and extensive unlabeled data for more robust and consistent registration and parcellation of longitudinal cortical surfaces. Our method utilizes the spherical topology characteristic of cortical surfaces. It employs a spherical network to function as an encoder, which extracts high-level cortical features. Subsequently, we build two specialized decoders dedicated to the tasks of registration and parcellation, respectively. To extract more meaningful spatial features, we design a novel parcellation map similarity loss to utilize the relationship between registration and parcellation tasks, i.e., the parcellation map warped by the deformation field in registration should match the atlas parcellation map, thereby providing extra supervision for the registration task and augmented data for parcellation task by warping the atlas parcellation map to unlabeled surfaces. To enable temporally more consistent feature representation, we additionally enforce longitudinal consistency among longitudinal surfaces after registering them together using their concatenated features. Experiments on two longitudinal datasets of infants and adults have shown that our method achieves significant improvements on both registration/parcellation accuracy and longitudinal consistency compared to existing methods, especially in small and challenging cortical regions.

SUGAR: Spherical ultrafast graph attention framework for cortical surface registration

Cortical surface registration plays a crucial role in aligning cortical functional and anatomical features across individuals. However, conventional registration algorithms are computationally inefficient. Recently, learning-based registration algorithms have emerged as a promising solution, significantly improving processing efficiency. Nonetheless, there remains a gap in the development of a learning-based method that exceeds the state-of-the-art conventional methods simultaneously in computational efficiency, registration accuracy, and distortion control, despite the theoretically greater representational capabilities of deep learning approaches. To address the challenge, we present SUGAR, a unified unsupervised deep-learning framework for both rigid and non-rigid registration. SUGAR incorporates a U-Net-based spherical graph attention network and leverages the Euler angle representation for deformation. In addition to the similarity loss, we introduce fold and multiple distortion losses to preserve topology and minimize various types of distortions. Furthermore, we propose a data augmentation strategy specifically tailored for spherical surface registration to enhance the registration performance. Through extensive evaluation involving over 10,000 scans from 7 diverse datasets, we showed that our framework exhibits comparable or superior registration performance in accuracy, distortion, and test-retest reliability compared to conventional and learning-based methods. Additionally, SUGAR achieves remarkable sub-second processing times, offering a notable speed-up of approximately 12,000 times in registering 9,000 subjects from the UK Biobank dataset in just 32 min. This combination of high registration performance and accelerated processing time may greatly benefit large-scale neuroimaging studies.

Automatic landmark detection and registration of brain cortical surfaces via quasi-conformal geometry and convolutional neural networks

In medical imaging, surface registration is extensively used for performing systematic comparisons between anatomical structures, with a prime example being the highly convoluted brain cortical surfaces. To obtain a meaningful registration, a common approach is to identify prominent features on the surfaces and establish a low-distortion mapping between them with the feature correspondence encoded as landmark constraints. Prior registration works have primarily focused on using manually labeled landmarks and solving highly nonlinear optimization problems, which are time-consuming and hence hinder practical applications. In this work, we propose a novel framework for the automatic landmark detection and registration of brain cortical surfaces using quasi-conformal geometry and convolutional neural networks. We first develop a landmark detection network (LD-Net) that allows for the automatic extraction of landmark curves given two prescribed starting and ending points based on the surface geometry. We then utilize the detected landmarks and quasi-conformal theory for achieving the surface registration. Specifically, we develop a coefficient prediction network (CP-Net) for predicting the Beltrami coefficients associated with the desired landmark-based registration and a mapping network called the disk Beltrami solver network (DBS-Net) for generating quasi-conformal mappings from the predicted Beltrami coefficients, with the bijectivity guaranteed by quasi-conformal theory. Experimental results are presented to demonstrate the effectiveness of our proposed framework. Altogether, our work paves a new way for surface-based morphometry and medical shape analysis.

Hierarchical particle optimization for cortical shape correspondence in temporal lobe resection

BackgroundAnterior temporal lobe resection is an effective treatment for temporal lobe epilepsy. The post-surgical structural changes could influence the follow-up treatment. Capturing post-surgical changes necessitates a well-established cortical shape correspondence between pre- and post-surgical surfaces. Yet, most cortical surface registration methods are designed for normal neuroanatomy. Surgical changes can introduce wide ranging artifacts in correspondence, for which conventional surface registration methods may not work as intended. MethodsIn this paper, we propose a novel particle method for one-to-one dense shape correspondence between pre- and post-surgical surfaces with temporal lobe resection. The proposed method can handle partial structural abnormality involving non-rigid changes. Unlike existing particle methods using implicit particle adjacency, we consider explicit particle adjacency to establish a smooth correspondence. Moreover, we propose hierarchical optimization of particles rather than full optimization of all particles at once to avoid trappings of locally optimal particle update. ResultsWe evaluate the proposed method on 25 pairs of T1-MRI with pre- and post-simulated resection on the anterior temporal lobe and 25 pairs of patients with actual resection. We show improved accuracy over several cortical regions in terms of ROI boundary Hausdorff distance with 4.29 mm and Dice similarity coefficients with average value 0.841, compared to existing surface registration methods on simulated data. In 25 patients with actual resection of the anterior temporal lobe, our method shows an improved shape correspondence in qualitative and quantitative evaluation on parcellation-off ratio with average value 0.061 and cortical thickness changes. We also show better smoothness of the correspondence without self-intersection, compared with point-wise matching methods which show various degrees of self-intersection. ConclusionThe proposed method establishes a promising one-to-one dense shape correspondence for temporal lobe resection. The resulting correspondence is smooth without self-intersection. The proposed hierarchical optimization strategy could accelerate optimization and improve the optimization accuracy. According to the results on the paired surfaces with temporal lobe resection, the proposed method outperforms the compared methods and is more reliable to capture cortical thickness changes.

Fast Spherical Mapping of Cortical Surface Meshes Using Deep Unsupervised Learning.

Spherical mapping of cortical surface meshes provides a more convenient and accurate space for cortical surface registration and analysis and thus has been widely adopted in neuroimaging field. Conventional approaches typically first inflate and project the original cortical surface mesh onto a sphere to generate an initial spherical mesh which contains large distortions. Then they iteratively reshape the spherical mesh to minimize the metric (distance), area or angle distortions. However, these methods suffer from two major issues: 1) the iterative optimization process is computationally expensive, making them not suitable for large-scale data processing; 2) when metric distortion cannot be further minimized, either area or angle distortion is minimized at the expense of the other, which is not flexible to generate application-specific meshes based on both of them. To address these issues, for the first time, we propose a deep learning-based algorithm to learn the mapping between the original cortical surface and spherical surface meshes. Specifically, we take advantage of the Spherical U-Net model to learn the spherical diffeomorphic deformation field for minimizing the distortions between the icosahedron-reparameterized original surface and spherical surface meshes. The end-to-end unsupervised learning scheme is very flexible to incorporate various optimization objectives. We further integrate it into a coarse-to-fine multi-resolution framework for better correcting fine-scaled distortions. We have validated our method on 800+ cortical surfaces, demonstrating reduced distortions than FreeSurfer (the most popularly used tool), while speeding up the process from 20 min to 5 s.

S3Reg: Superfast Spherical Surface Registration Based on Deep Learning.

Cortical surface registration is an essential step and prerequisite for surface-based neuroimaging analysis. It aligns cortical surfaces across individuals and time points to establish cross-sectional and longitudinal cortical correspondences to facilitate neuroimaging studies. Though achieving good performance, available methods are either time consuming or not flexible to extend to multiple or high dimensional features. Considering the explosive availability of large-scale and multimodal brain MRI data, fast surface registration methods that can flexibly handle multimodal features are desired. In this study, we develop a Superfast Spherical Surface Registration (S3Reg) framework for the cerebral cortex. Leveraging an end-to-end unsupervised learning strategy, S3Reg offers great flexibility in the choice of input feature sets and output similarity measures for registration, and meanwhile reduces the registration time significantly. Specifically, we exploit the powerful learning capability of spherical Convolutional Neural Network (CNN) to directly learn the deformation fields in spherical space and implement diffeomorphic design with "scaling and squaring" layers to guarantee topology-preserving deformations. To handle the polar-distortion issue, we construct a novel spherical CNN model using three orthogonal Spherical U-Nets. Experiments are performed on two different datasets to align both adult and infant multimodal cortical features. Results demonstrate that our S3Reg shows superior or comparable performance with state-of-the-art methods, while improving the registration time from 1 min to 10 sec.

A Deep Network for Joint Registration and Parcellation of Cortical Surfaces.

Cortical surface registration and parcellation are two essential steps in neuroimaging analysis. Conventionally, they are performed independently as two tasks, ignoring the inherent connections of these two closely-related tasks. Essentially, both tasks rely on meaningful cortical feature representations, so they can be jointly optimized by learning shared useful cortical features. To this end, we propose a deep learning framework for joint cortical surface registration and parcellation. Specifically, our approach leverages the spherical topology of cortical surfaces and uses a spherical network as the shared encoder to first learn shared features for both tasks. Then we train two task-specific decoders for registration and parcellation, respectively. We further exploit the more explicit connection between them by incorporating the novel parcellation map similarity loss to enforce the boundary consistency of regions, thereby providing extra supervision for the registration task. Conversely, parcellation network training also benefits from the registration, which provides a large amount of augmented data by warping one surface with manual parcellation map to another surface, especially when only few manually-labeled surfaces are available. Experiments on a dataset with more than 600 cortical surfaces show that our approach achieves large improvements on both parcellation and registration accuracy (over separately trained networks) and enables training high-quality parcellation and registration models using much fewer labeled data.

Learning 4D Infant Cortical Surface Atlas with Unsupervised Spherical Networks.

Spatiotemporal (4D) cortical surface atlas during infancy plays an important role for surface-based visualization, normalization and analysis of the dynamic early brain development. Conventional atlas construction methods typically rely on classical group-wise registration on sub-populations and ignore longitudinal constraints, thus having three main issues: 1) constructing templates at discrete time points; 2) resulting in longitudinal inconsistency among different age's atlases; and 3) taking extremely long runtime. To address these issues, in this paper, we propose a fast unsupervised learning-based surface atlas construction framework incorporating longitudinal constraints to enforce the within-subject temporal correspondence in the atlas space. To well handle the difficulties of learning large deformations, we propose a multi-level multimodal spherical registration network to perform cortical surface registration in a coarse-to-fine manner. Thus, only small deformations need to be estimated at each resolution level using the registration network, which further improves registration accuracy and atlas quality. Our constructed 4D infant cortical surface atlas based on 625 longitudinal scans from 291 infants is temporally continuous, in contrast to the state-of-the-art UNC 4D Infant Surface Atlas, which only provides the atlases at a few discrete sparse time points. By evaluating the intra- and inter-subject spatial normalization accuracy after alignment onto the atlas, our atlas demonstrates more detailed and fine-grained cortical patterns, thus leading to higher accuracy in surface registration.

Improving a probabilistic cytoarchitectonic atlas of auditory cortex using a novel method for inter-individual alignment.

The human superior temporal plane, the site of the auditory cortex, displays high inter-individual macro-anatomical variation. This questions the validity of curvature-based alignment (CBA) methods for in vivo imaging data. Here, we have addressed this issue by developing CBA+, which is a cortical surface registration method that uses prior macro-anatomical knowledge. We validate this method by using cytoarchitectonic areas on 10 individual brains (which we make publicly available). Compared to volumetric and standard surface registration, CBA+ results in a more accurate cytoarchitectonic auditory atlas. The improved correspondence of micro-anatomy following the improved alignment of macro-anatomy validates the superiority of CBA+ compared to CBA. In addition, we use CBA+ to align in vivo and postmortem data. This allows projection of functional and anatomical information collected in vivo onto the cytoarchitectonic areas, which has the potential to contribute to the ongoing debate on the parcellation of the human auditory cortex.

Cortical surface registration using unsupervised learning

Non-rigid cortical registration is an important and challenging task due to the geometric complexity of the human cortex and the high degree of inter-subject variability. A conventional solution is to use a spherical representation of surface properties and perform registration by aligning cortical folding patterns in that space. This strategy produces accurate spatial alignment, but often requires high computational cost. Recently, convolutional neural networks (CNNs) have demonstrated the potential to dramatically speed up volumetric registration. However, due to distortions introduced by projecting a sphere to a 2D plane, a direct application of recent learning-based methods to surfaces yields poor results. In this study, we present SphereMorph, a diffeomorphic registration framework for cortical surfaces using deep networks that addresses these issues. SphereMorph uses a UNet-style network associated with a spherical kernel to learn the displacement field and warps the sphere using a modified spatial transformer layer. We propose a resampling weight in computing the data fitting loss to account for distortions introduced by polar projection, and demonstrate the performance of our proposed method on two tasks, including cortical parcellation and group-wise functional area alignment. The experiments show that the proposed SphereMorph is capable of modeling the geometric registration problem in a CNN framework and demonstrate superior registration accuracy and computational efficiency. The source code of SphereMorph will be released to the public upon acceptance of this manuscript at https://github.com/voxelmorph/spheremorph.

AUTOMATIC LABELING OF CORTICAL SULCI USING SPHERICAL CONVOLUTIONAL NEURAL NETWORKS IN A DEVELOPMENTAL COHORT.

In this paper, we present the automatic labeling framework for sulci in the human lateral prefrontal cortex (PFC). We adapt an existing spherical U-Net architecture with our recent surface data augmentation technique to improve the sulcal labeling accuracy in a developmental cohort. Specifically, our framework consists of the following key components: (1) augmented geometrical features being generated during cortical surface registration, (2) spherical U-Net architecture to efficiently fit the augmented features, and (3) postrefinement of sulcal labeling by optimizing spatial coherence via a graph cut technique. We validate our method on 30 healthy subjects with manual labeling of sulcal regions within PFC. In the experiments, we demonstrate significantly improved labeling performance (0.7749) in mean Dice overlap compared to that of multi-atlas (0.6410) and standard spherical U-Net (0.7011) approaches, respectively (p < 0.05). Additionally, the proposed method achieves a full set of sulcal labels in 20 seconds in this developmental cohort.

DIFFEOMORPHIC REGISTRATION FOR RETINOTOPIC MAPPING VIA QUASICONFORMAL MAPPING.

Human visual cortex is organized into several functional regions/areas. Identifying these visual areas of the human brain (i.e., V1, V2, V4, etc) is an important topic in neurophysiology and vision science. Retinotopic mapping via functional magnetic resonance imaging (fMRI) provides a non-invasive way of defining the boundaries of the visual areas. It is well known from neurophysiology studies that retinotopic mapping is diffeomorphic within each local area (i.e. locally smooth, differentiable, and invertible). However, due to the low signal-noise ratio of fMRI, the retinotopic maps from fMRI are often not diffeomorphic, making it difficult to delineate the boundaries of visual areas. The purpose of this work is to generate diffeomorphic retinotopic maps and improve the accuracy of the retinotopic atlas from fMRI measurements through the development of a specifically designed registration procedure. Although there are sophisticated existing cortical surface registration methods, most of them cannot fully utilize the features of retinotopic mapping. By considering unique retinotopic mapping features, we form a quasiconformal geometry-based registration model and solve it with efficient numerical methods. We compare our registration with several popular methods on synthetic data. The results demonstrate that the proposed registration is superior to conventional methods for the registration of retinotopic maps. The application of our method to a real retinotopic mapping dataset also results in much smaller registration errors.

Unsupervised Learning for Spherical Surface Registration.

Current spherical surface registration methods achieve good performance on alignment and spatial normalization of cortical surfaces across individuals in neuroimaging analysis. However, they are computationally intensive, since they have to optimize an objective function independently for each pair of surfaces. In this paper, we present a fast learning-based algorithm that makes use of the recent development in spherical Convolutional Neural Networks (CNNs) for spherical cortical surface registration. Given a set of surface pairs without supervised information such as ground truth deformation fields or anatomical landmarks, we formulate the registration as a parametric function and learn its parameters by enforcing the feature similarity between one surface and the other one warped by the estimated deformation field using the function. Then, given a new pair of surfaces, we can quickly infer the spherical deformation field registering one surface to the other one. We model this parametric function using three orthogonal Spherical U-Nets and use spherical transform layers to warp the spherical surfaces, while imposing smoothness constraints on the deformation field. All the layers in the network are well-defined and differentiable, thus the parameters can be effectively learned. We show that our method achieves accurate cortical alignment results on 102 subjects, comparable to two state-of-the-art methods: Spherical Demons and MSM, while runs much faster.

Development and evaluation of a "trackerless" surgical planning and guidance system based on 3D Slicer.

Conventional optical tracking systems use cameras sensitive to near-infrared (NIR) light and NIR illuminated/active-illuminating markers to localize instrumentation and the patient in the operating room (OR) physical space. This technology is widely used within the neurosurgical theater and is a staple in the standard of care for craniotomy planning. To accomplish, planning is largely conducted at the time of the procedure in the OR with the patient in a fixed head orientation. We propose a framework to achieve this in the OR without conventional tracking technology, i.e., a "trackerless" approach. Briefly, we investigate an extension of the 3D Slicer which combines surgical planning and craniotomy designation. While taking advantage of the well-developed 3D Slicer platform, we implement advanced features to aid the neurosurgeon in planning the location of the anticipated craniotomy relative to the preoperatively imaged tumor in a physical-to-virtual setup, and then subsequently aid the true physical procedure by correlating that physical-to-virtual plan with an intraoperative magnetic resonance imaging-to-physical registered field-of-view display. These steps are done such that the craniotomy can be designated without the use of a conventional optical tracking technology. To test this approach, four experienced neurosurgeons performed experiments on five different surgical cases using our 3D Slicer module as well as the conventional procedure for comparison. The results suggest that our planning system provides a simple, cost-efficient, and reliable solution for surgical planning and delivery without the use of conventional tracking technologies. We hypothesize that the combination of this craniotomy planning approach and our past developments in cortical surface registration and deformation tracking using stereo-pair data from the surgical microscope may provide a fundamental realization of an integrated trackerless surgical guidance platform.

Hierarchical spherical deformation for cortical surface registration.

We present hierarchical spherical deformation for a group-wise shape correspondence to address template selection bias and to minimize registration distortion. In this work, we aim at a continuous and smooth deformation field to guide accurate cortical surface registration. In conventional spherical registration methods, a global rigid alignment and local deformation are independently performed. Motivated by the composition of precession and intrinsic rotation, we simultaneously optimize global rigid rotation and non-rigid local deformation by utilizing spherical harmonics interpolation of local composite rotations in a single framework. To this end, we indirectly encode local displacements by such local composite rotations as functions of spherical locations. Furthermore, we introduce an additional regularization term to the spherical deformation, which maximizes its rigidity while reducing registration distortion. To improve surface registration performance, we employ the second order approximation of the energy function that enables fast convergence of the optimization. In the experiments, we validate our method on healthy normal subjects with manual cortical surface parcellation in registration accuracy and distortion. We show an improved shape correspondence with high accuracy in cortical surface parcellation and significantly low registration distortion in surface area and edge length. In addition to validation, we discuss parameter tuning, optimization, and implementation design with potential acceleration.

Construction of 4D infant cortical surface atlases with sharp folding patterns via spherical patch‐based group‐wise sparse representation

4D (spatial + temporal) infant cortical surface atlases covering dense time points are highly needed for understanding dynamic early brain development. In this article, we construct a set of 4D infant cortical surface atlases with longitudinally consistent and sharp cortical attribute patterns at 11 time points in the first six postnatal years, that is, at 1, 3, 6, 9, 12, 18, 24, 36, 48, 60, and 72 months of age, which is targeted for better normalization of the dynamic changing early brain cortical surfaces. To ensure longitudinal consistency and unbiasedness, we adopt a two-stage group-wise surface registration. To preserve sharp cortical attribute patterns on the atlas, instead of simply averaging over the coregistered cortical surfaces, we leverage a spherical patch-based sparse representation using the augmented dictionary to overcome the potential registration errors. Our atlases provide not only geometric attributes of the cortical folding, but also cortical thickness and myelin content. Therefore, to address the consistency across different cortical attributes on the atlas, instead of sparsely representing each attribute independently, we jointly represent all cortical attributes with a group-wise sparsity constraint. In addition, to further facilitate region-based analysis using our atlases, we have also provided two widely used parcellations, that is, FreeSurfer parcellation and multimodal parcellation, on our 4D infant cortical surface atlases. Compared to cortical surface atlases constructed with other methods, our cortical surface atlases preserve sharper cortical folding attribute patterns, thus leading to better accuracy in registration of individual infant cortical surfaces to the atlas.

Hierarchical spherical deformation for cortical surface registration

Hierarchical spherical deformation for cortical surface registration

CONSTRUCTION OF SPATIOTEMPORAL INFANT CORTICAL SURFACE ATLAS OF RHESUS MACAQUE.

As a widely used animal model in MR imaging studies, rhesus macaque helps to better understand both normal and abnormal neural development in the human brain. However, the available adult macaque brain atlases are not well suitable for study of brain development at the early postnatal stage, since this stage undergoes dramatic changes in brain appearances and structures. Building age matched atlases for this critical period is thus highly desirable yet still lacking. In this paper, we construct the first spatiotemporal (4D) cortical surface atlases for rhesus macaques from 2 weeks to 24 months, using 138 longitudinal MRI scans from 32 healthy rhesus monkeys. Specifically, we first perform intra-subject cortical surface registration to obtain within-subject mean cortical surfaces. Then, we perform inter-subject registration of within-subject mean surfaces to obtain unbiased and longitudinally-consistent 4D cortical surface atlases. Based on our 4D rhesus monkey atlases, we further chart the first developmental-trajectories-based parcellation maps using the local surface area and spectral clustering algorithm. Our 4D macaque surface atlases and parcellation maps will greatly facilitate early brain development studies of macaques.

RfDemons: Resting fMRI-based Cortical Surface Registration using the BrainSync Transform.

Cross subject functional studies of cerebral cortex require cortical registration that aligns functional brain regions. While cortical folding patterns are approximate indicators of the underlying cytoarchitecture, coregistration based on these features alone does not accurately align functional regions in cerebral cortex. This paper presents a method for cortical surface registration (rfDemons) based on resting fMRI (rfMRI) data that uses curvature-based anatomical registration as an initialization. In contrast to existing techniques that use connectivity-based features derived from rfMRI, the proposed method uses 'synchronized' resting rfMRI time series directly. The synchronization of rfMRI data is performed using the BrainSync transform which applies an orthogonal transform to the rfMRI time series to temporally align them across subjects. The rfDemons method was applied to rfMRI from the Human Connectome Project and evaluated using task fMRI data to explore the impact of cortical registration performed using resting fMRI data on functional alignment of the cerebral cortex.

Registration-Free Infant Cortical Surface Parcellation using Deep Convolutional Neural Networks.

Automatic parcellation of infant cortical surfaces into anatomical regions of interest (ROIs) is of great importance in brain structural and functional analysis. Conventional cortical surface parcellation methods suffer from two main issues: 1) Cortical surface registration is needed for establishing the atlas-to-individual correspondences; 2) The mapping from cortical shape to the parcellation labels requires designing of specific hand-crafted features. To address these issues, in this paper, we propose a novel cortical surface parcellation method, which is free of surface registration and designing of hand-crafted features, based on deep convolutional neural network (DCNN). Our main idea is to formulate surface parcellation as a patch-wise classification problem. Briefly, we use DCNN to train a classifier, whose inputs are the local cortical surface patches with multi-channel cortical shape descriptors such as mean curvature, sulcal depth, and average convexity; while the outputs are the parcellation label probabilities of cortical vertices. To enable effective convolutional operation on the surface data, we project each spherical surface patch onto its intrinsic tangent plane by a geodesic-distance-preserving mapping. Then, after classification, we further adopt the graph cuts method to improve spatial consistency of the parcellation. We have validated our method based on 90 neonatal cortical surfaces with manual parcellations, showing superior accuracy and efficiency of our proposed method.