Underlying Image Data Research Articles

OpenHands: An Open-Source Statistical Shape Model of the Finger Bones

This paper presents statistical shape models of the four fingers of the hand, with an emphasis on anatomic analysis of the proximal and distal interphalangeal joints. A multi-body statistical shape modelling pipeline was implemented on an exemplar training dataset of computed tomography (CT) scans of 10 right hands (5F:5M, 27–37 years, free from disease or injury) imaged at 0.3 mm resolution, segmented, meshed and aligned. Model generated included pose neutralisation to remove joint angle variation during imaging. Repositioning was successful; no joint flexion variation was observed in the resulting model. The first principal component (PC) of morphological variation represented phalanx size in all fingers. Subsequent PCs showed variation in position along the palmar-dorsal axis, and bone breadth: length ratio. Finally, the models were interrogated to provide gross measures of bone lengths and joint spaces. These models have been published for open use to support wider community efforts in hand biomechanical analysis, providing bony anatomy descriptions whilst preserving the security of the underlying imaging data and privacy of the participants. The model describes a small, homogeneous population, and assumptions cannot be made about how it represents individuals outside the training dataset. However, it supplements anthropometric datasets with additional shape information, and may be useful for investigating factors such as joint morphology and design of hand-interfacing devices and products. The model has been shared as an open-source repository (https://github.com/abel-research/OpenHands), and we encourage the community to use and contribute to it.

VisuStatR: visualizing motility and morphology statistics on images in R.

Live-cell microscopy has become an essential tool for analyzing dynamic processes in various biological applications. Thereby, high-throughput and automated tracking analyses allow the simultaneous evaluation of large numbers of objects. However, to critically assess the influence of individual objects on calculated summary statistics, and to detect heterogeneous dynamics or possible artifacts, such as misclassified or -tracked objects, a direct mapping of gained statistical information onto the actual image data would be necessary. We present VisuStatR as a platform independent software package that allows the direct visualization of time-resolved summary statistics of morphological characteristics or motility dynamics onto raw images. The software contains several display modes to compare user-defined summary statistics and the underlying image data in various levels of detail. VisuStatR is a free and open-source R-package, containing a user-friendly graphical-user interface and is available via GitHub at https://github.com/grrchrr/VisuStatR/ under the MIT+ license. Supplementary data are available at Bioinformatics online.

Constructing and optimizing 3D atlases from 2D data with application to the developing mouse brain.

3D imaging data necessitate 3D reference atlases for accurate quantitative interpretation. Existing computational methods to generate 3D atlases from 2D-derived atlases result in extensive artifacts, while manual curation approaches are labor-intensive. We present a computational approach for 3D atlas construction that substantially reduces artifacts by identifying anatomical boundaries in the underlying imaging data and using these to guide 3D transformation. Anatomical boundaries also allow extension of atlases to complete edge regions. Applying these methods to the eight developmental stages in the Allen Developing Mouse Brain Atlas (ADMBA) led to more comprehensive and accurate atlases. We generated imaging data from 15 whole mouse brains to validate atlas performance and observed qualitative and quantitative improvement (37% greater alignment between atlas and anatomical boundaries). We provide the pipeline as the MagellanMapper software and the eight 3D reconstructed ADMBA atlases. These resources facilitate whole-organ quantitative analysis between samples and across development.

Introducing Biomedisa as an open-source online platform for biomedical image segmentation

We present Biomedisa, a free and easy-to-use open-source online platform developed for semi-automatic segmentation of large volumetric images. The segmentation is based on a smart interpolation of sparsely pre-segmented slices taking into account the complete underlying image data. Biomedisa is particularly valuable when little a priori knowledge is available, e.g. for the dense annotation of the training data for a deep neural network. The platform is accessible through a web browser and requires no complex and tedious configuration of software and model parameters, thus addressing the needs of scientists without substantial computational expertise. We demonstrate that Biomedisa can drastically reduce both the time and human effort required to segment large images. It achieves a significant improvement over the conventional approach of densely pre-segmented slices with subsequent morphological interpolation as well as compared to segmentation tools that also consider the underlying image data. Biomedisa can be used for different 3D imaging modalities and various biomedical applications.

Measurement of PET Quantitative Bias In Vivo.

Quantitative imaging biomarkers are widely used in PET for both research and clinical applications, yet bias in the underlying image data has not been well characterized. In the absence of a readily available reference standard for in vivo quantification, bias in PET images has been inferred using physical phantoms, even though arrangements of this sort provide only a poor approximation of the imaging environment in real patient examinations. In this study, we used data acquired from patient volunteers to assess PET quantitative bias in vivo. Image-derived radioactivity concentrations in the descending aorta were compared with blood samples counted on a calibrated γ-counter. Methods: Ten patients with prostate cancer were studied using 2-(3-(1-carboxy-5-[(6-18F-fluoro-pyridine-3-carbonyl)-amino]-pentyl)-ureido)-pentanedioic acid PET/CT. For each patient, 3 whole-body PET/CT image series were acquired after a single administration of the radiotracer: shortly after injection as well as approximately 1 and 4 h later. Venous blood samples were obtained at 8 time points over an 8-h period, and whole blood was counted on a NaI γ-counter. A 10-mm-diameter, 20-mm-long cylindric volume of interest was positioned in the descending thoracic aorta to estimate the PET-derived radioactivity concentration in blood. A triexponential function was fit to the γ-counter blood data and used to estimate the radioactivity concentration at the time of each PET acquisition. Results: The PET-derived and γ-counter-derived radioactivity concentrations were linearly related, with an R2 of 0.985, over a range of relevant radioactivity concentrations. The mean difference between the PET and γ-counter data was 4.8% ± 8.6%, with the PET measurements tending to be greater. Conclusion: Human image data acquired on a conventional whole-body PET/CT system with a typical clinical protocol differed by an average of around 5% from blood samples counted on a calibrated γ-counter. This bias may be partly attributable to residual uncorrected scatter or attenuation correction error. These data offer an opportunity for the assessment of PET bias in vivo and provide additional support for the use of quantitative imaging biomarkers.

An Uncertainty-aware Workflow for Keyhole Surgery Planning using Hierarchical Image Semantics

Keyhole surgeries become increasingly important in clinical daily routine as they help minimizing the damage of a patient’s healthy tissue. The planning of keyhole surgeries is based on medical imaging and an important factor that influences the surgeries’ success. Due to the image reconstruction process, medical image data contains uncertainty that exacerbates the planning of a keyhole surgery. In this paper we present a visual workflow that helps clinicians to examine and compare different surgery paths as well as visualizing the patients’ affected tissue. The analysis is based on the concept of hierarchical image semantics, that segment the underlying image data with respect to the input images’ uncertainty and the users understanding of tissue composition. Users can define arbitrary surgery paths that they need to investigate further. The defined paths can be queried by a rating function to identify paths that fulfill user-defined properties. The workflow allows a visual inspection of the affected tissues and its substructures. Therefore, the workflow includes a linked view system indicating the three-dimensional location of selected surgery paths as well as how these paths affect the patients tissue. To show the effectiveness of the presented approach, we applied it to the planning of a keyhole surgery of a brain tumor removal and a kneecap surgery.

Multivariate statistical analysis of diffusion imaging parameters using partial least squares: Application to white matter variations in Alzheimer's disease

Diffusion magnetic resonance imaging (dMRI) is a unique technology that allows the noninvasive quantification of microstructural tissue properties of the human brain in healthy subjects as well as the probing of disease-induced variations. Population studies of dMRI data have been essential in identifying pathological structural changes in various conditions, such as Alzheimer's and Huntington's diseases (Salat et al., 2010; Rosas et al., 2006). The most common form of dMRI involves fitting a tensor to the underlying imaging data (known as diffusion tensor imaging, or DTI), then deriving parametric maps, each quantifying a different aspect of the underlying microstructure, e.g. fractional anisotropy and mean diffusivity. To date, the statistical methods utilized in most DTI population studies either analyzed only one such map or analyzed several of them, each in isolation. However, it is most likely that variations in the microstructure due to pathology or normal variability would affect several parameters simultaneously, with differing variations modulating the various parameters to differing degrees. Therefore, joint analysis of the available diffusion maps can be more powerful in characterizing histopathology and distinguishing between conditions than the widely used univariate analysis. In this article, we propose a multivariate approach for statistical analysis of diffusion parameters that uses partial least squares correlation (PLSC) analysis and permutation testing as building blocks in a voxel-wise fashion. Stemming from the common formulation, we present three different multivariate procedures for group analysis, regressing-out nuisance parameters and comparing effects of different conditions. We used the proposed procedures to study the effects of non-demented aging, Alzheimer's disease and mild cognitive impairment on the white matter. Here, we present results demonstrating that the proposed PLSC-based approach can differentiate between effects of different conditions in the same region as well as uncover spatial variations of effects across the white matter. The proposed procedures were able to answer questions on structural variations such as: “are there regions in the white matter where Alzheimer's disease has a different effect than aging or similar effect as aging?” and “are there regions in the white matter that are affected by both mild cognitive impairment and Alzheimer's disease but with differing multivariate effects?”

MULTI-RESOLUTION STATISTICAL ANALYSIS ON GRAPH STRUCTURED DATA IN NEUROIMAGING.

Statistical data analysis plays a major role in discovering structural and functional imaging phenotypes for mental disorders such as Alzheimer's disease (AD). The goal here is to identify, ideally early on, which regions in the brain show abnormal variations with a disorder. To make the method more sensitive, we rely on a multi-resolutional perspective of the given data. Since the underlying imaging data (such as cortical surfaces and connectomes) are naturally represented in the form of weighted graphs which lie in a non-Euclidean space, we introduce recent work from the harmonics literature to derive an effective multi-scale descriptor using wavelets on graphs that characterize the local context at each data point. Using this descriptor, we demonstrate experiments where we identify significant differences between AD and control populations using cortical surface data and tractography derived graphs/networks.

Methods and applications for cardiovascular 3D-printing

Background: Three-dimensional printing is rapidly growing field that enables the exact replication of anatomical structures in a physical model. 3D printing may be used for cardiothoracic procedural planning, education and a variety of research purposes. Multiple 3D printer designs exist and the technological requirements may be complex for clinician scientists. Methods: Using high quality imaging data from multiple modalities we have 3d-printed a range of cardiovascular structures using different printing techniques in order to assess their suitability for cardiovascular applications. We isolated and generated 3D meshes of a whole heart (gated cardiac CT), aortic coarctation (cardiac MRI), mitral valve (3D echocardiography) and left atrium (cardiac CT). We used fusedfilament fabrication (FFF), polymer jet, laser stereolithographic (SLA) and powder bed 3D printers to print selected structures. The advantages and limitation of each printing method was assessed. Results: 3D printing is highly contingent on the quality of the underlying imaging data and image post-processing is required. 3D mesh generation can be difficult and care needs to be taken to preserve accuracy during data manipulation. Printing techniques vary widely with regards to minimal resolution, colour reproduction, speed, cost and geometric limitations. Conclusion: 3D printing is a useful and feasible technique for displaying three-dimensional cardiac imaging. The printing technique used should be tailored to the application. Current technology allows 3D printing to be widely implemented.

Inexact matching of structural models based on the duality of patterns and classifiers

In many computer vision problems, the essential information can be most easily interpreted in the form of structural models. However, the computation of distances between structural models can be difficult, since small changes in the underlying image data often cause significant differences in the graph layout. The other way around, changes in the link structure of a graph often mean complex changes in feature attributes or do not correspond to any valid visual data at all. In contrast, structural models can often be used conveniently for the classification of a pattern. Based on this observation, we propose to shift the graph matching problem to the easier problem of matching classifiers and patterns. The similarity between two models is detected if a model serving as a pattern can be recognised by another model serving as classifier. To extend this measure beyond single binary digits, models are compared to whole sets of other models. Single models are described by the vector of similarities to the set. Further comparisons between models can be done using appropriate distance functions on the vector representation. The use of the method is demonstrated in a graph clustering task. We also discuss the numerical stability of the method with respect to the dimensionality of the descriptors.

VAICo: Visual Analysis for Image Comparison

Scientists, engineers, and analysts are confronted with ever larger and more complex sets of data, whose analysis poses special challenges. In many situations it is necessary to compare two or more datasets. Hence there is a need for comparative visualization tools to help analyze differences or similarities among datasets. In this paper an approach for comparative visualization for sets of images is presented. Well-established techniques for comparing images frequently place them side-by-side. A major drawback of such approaches is that they do not scale well. Other image comparison methods encode differences in images by abstract parameters like color. In this case information about the underlying image data gets lost. This paper introduces a new method for visualizing differences and similarities in large sets of images which preserves contextual information, but also allows the detailed analysis of subtle variations. Our approach identifies local changes and applies cluster analysis techniques to embed them in a hierarchy. The results of this process are then presented in an interactive web application which allows users to rapidly explore the space of differences and drill-down on particular features. We demonstrate the flexibility of our approach by applying it to multiple distinct domains.

Automatic lumen and outer wall segmentation of the carotid artery using deformable three‐dimensional models in MR angiography and vessel wall images

To develop and validate an automated segmentation technique for the detection of the lumen and outer wall boundaries in MR vessel wall studies of the common carotid artery. A new segmentation method was developed using a three-dimensional (3D) deformable vessel model requiring only one single user interaction by combining 3D MR angiography (MRA) and 2D vessel wall images. This vessel model is a 3D cylindrical Non-Uniform Rational B-Spline (NURBS) surface which can be deformed to fit the underlying image data. Image data of 45 subjects was used to validate the method by comparing manual and automatic segmentations. Vessel wall thickness and volume measurements obtained by both methods were compared. Substantial agreement was observed between manual and automatic segmentation; over 85% of the vessel wall contours were segmented successfully. The interclass correlation was 0.690 for the vessel wall thickness and 0.793 for the vessel wall volume. Compared with manual image analysis, the automated method demonstrated improved interobserver agreement and inter-scan reproducibility. Additionally, the proposed automated image analysis approach was substantially faster. This new automated method can reduce analysis time and enhance reproducibility of the quantification of vessel wall dimensions in clinical studies.

A Remote-Sensing Method of Selecting Reference Stations for Evaluating Urbanization Effect on Surface Air Temperature Trends

Abstract In the global lands, the bias of urbanization effects still exits in the surface air temperature series of many city weather stations to a certain extent. Reliable reference climate stations need to be selected for the detection and correction of the local manmade warming bias. The underlying image data of remote sensing retrieval is adopted in this study to obtain the spatial distribution of surface brightness temperature, and the surface air temperature reference stations are determined based on the locations of the weather stations in the remote sensing surface thermal fields. Among the 672 national reference climate stations and national basic weather stations of mainland China, for instance, 113 surface air temperature reference stations are selected for applying this method. Compared with the average surface air temperature series of the reference stations obtained by a more sophisticated method developed in China, this method is proven to be robust and applicable, and can be adopted for the evaluation and adjustment study on the urbanization bias of the currently used air temperature records of surface climate stations in the global lands.

The DIADEM Metric: Comparing Multiple Reconstructions of the Same Neuron

Digital reconstructions of neuronal morphology are used to study neuron function, development, and responses to various conditions. Although many measures exist to analyze differences between neurons, none is particularly suitable to compare the same arborizing structure over time (morphological change) or reconstructed by different people and/or software (morphological error). The metric introduced for the DIADEM (DIgital reconstruction of Axonal and DEndritic Morphology) Challenge quantifies the similarity between two reconstructions of the same neuron by matching the locations of bifurcations and terminations as well as their topology between the two reconstructed arbors. The DIADEM metric was specifically designed to capture the most critical aspects in automating neuronal reconstructions, and can function in feedback loops during algorithm development. During the Challenge, the metric scored the automated reconstructions of best-performing algorithms against manually traced gold standards over a representative data set collection. The metric was compared with direct quality assessments by neuronal reconstruction experts and with clocked human tracing time saved by automation. The results indicate that relevant morphological features were properly quantified in spite of subjectivity in the underlying image data and varying research goals. The DIADEM metric is freely released open source ( http://diademchallenge.org ) as a flexible instrument to measure morphological error or change in high-throughput reconstruction projects.

Patient-Specific Planning for Radio-Frequency Ablation of Tumors in the Presence of Uncertainty

Abstract The radio frequency (RF) ablation is a promising minimally invasive form of treatment for hepatic metastases and primary tumors. Thereby a needle like applicator, which is interstitially placed into the lesion, induces an electric current that causes heating and consequent destruction of the tissue due to its Ohmic resistance. In order to be a true alternative to the standard surgical resection, RF ablation must lead to a result similar to R0 resections. Here, patient specific mathematical modeling and numerical simulation of the bio-physical processes lead to a valuable support of the therapy planning, because they allow for an a priori estimation of the success as well as an optimization of the therapy parameters. In this article we discuss a mathematical model of partial differential equations (PDEs) for the patient specific numerical simulation of RF ablation. A particular focus lies on the consideration of uncertainties in the material properties, the underlying image data as well as the computational results. We discuss a stochastic PDE model, which allows for a sensitivity analysis of the computational results with respect to perturbations in the material properties. Furthermore a method for the fast estimation of the thermal necrosis is shown, which bases on the separation of non patient specific pre-calculations and patient specific computations, leading to an interactive real-time simulation tool.

Virtual cardiotomy based on 3-D MRI for preoperative planning in congenital heart disease

Patient-specific preoperative planning in complex congenital heart disease may be greatly facilitated by virtual cardiotomy. Surgeons can perform an unlimited number of surgical incisions on a virtual 3-D reconstruction to evaluate the feasibility of different surgical strategies. To quantitatively evaluate the quality of the underlying imaging data and the accuracy of the corresponding segmentation, and to qualitatively evaluate the feasibility of virtual cardiotomy. A whole-heart MRI sequence was applied in 42 children with congenital heart disease (age 3 +/- 3 years, weight 13 +/- 9 kg, heart rate 96 +/- 21 bpm). Image quality was graded 1-4 (diagnostic image quality > or =2) by two independent blinded observers. In patients with diagnostic image quality the segmentation quality was also graded 1-4 (4 no discrepancies, 1 misleading error). The average image quality score was 2.7 - sufficient for virtual reconstruction in 35 of 38 patients (92%) older than 1 month. Segmentation time was 59 +/- 10 min (average quality score 3.5). Virtual cardiotomy was performed in 19 patients. Accurate virtual reconstructions of patient-specific cardiac anatomy can be produced in less than 1 h from 3-D MRI. The presented work thus introduces a new, clinically feasible noninvasive technique for improved preoperative planning in complex cases of congenital heart disease.

Composite finite elements for 3D image based computing

We present an algorithmical concept for modeling and simulation with partial differential equations (PDEs) in image based computing where the computational geometry is defined through previously segmented image data. Such problems occur in applications from biology and medicine where the underlying image data has been acquired through, e.g. computed tomography (CT), magnetic resonance imaging (MRI) or electron microscopy (EM). Based on a level-set description of the computational domain, our approach is capable of automatically providing suitable composite finite element functions that resolve the complicated shapes in the medical/biological data set. It is efficient in the sense that the traversal of the grid (and thus assembling matrices for finite element computations) inherits the efficiency of uniform grids away from complicated structures. The method’s efficiency heavily depends on precomputed lookup tables in the vicinity of the domain boundary or interface. A suitable multigrid method is used for an efficient solution of the systems of equations resulting from the composite finite element discretization. The paper focuses on both algorithmical and implementational details. Scalar and vector valued model problems as well as real applications underline the usability of our approach.

Local or Global Minima: Flexible Dual-Front Active Contours

Most variational active contour models are designed to find local minima of data-dependent energy functionals with the hope that reasonable initial placement of the active contour will drive it toward a "desirable" local minimum as opposed to an undesirable configuration due to noise or complex image structure. As such, there has been much research into the design of complex region-based energy functionals that are less likely to yield undesirable local minima when compared to simpler edge-based energy functionals whose sensitivity to noise and texture is significantly worse. Unfortunately, most of these more "robust" region-based energy functionals are applicable to a much narrower class of imagery compared to typical edge-based energies due to stronger global assumptions about the underlying image data. Devising new implementation algorithms for active contours that attempt to capture more global minimizers of already proposed image-based energies would allow us to choose an energy that makes sense for a particular class of energy without concern over its sensitivity to local minima. Such implementations have been proposed for capturing global minima. However, sometimes the completely-global minimum is just as undesirable as a minimum that is too local. In this paper, we propose a novel, fast, and flexible dual front implementation of active contours, motivated by minimal path techniques and utilizing fast sweeping algorithms, which is easily manipulated to yield minima with variable "degrees" of localness and globalness. By simply adjusting the size of active regions, the ability to gracefully move from capturing minima that are more local (according to the initial placement of the active contour/surface) to minima that are more global allows this model to more easily obtain "desirable" minimizers (which often are neither the most local nor the most global). Experiments on various 2D and 3D images and comparisons with some active contour models and region-growing methods are also given to illustrate the properties of this model and its performance in a variety of segmentation applications.

Diagnostic medical ultrasound system having method and apparatus for storing and retrieving 3D and 4D data sets

A method and system is disclosed which permits a user to save user defined viewing configuration, also referred to as a bookmark as a separately manipulatable or transferable entity without altering the underlying image data and maintaining, where desired, the association between the bookmark 200 and the underlying source image data. The disclosed system and method act to save the bookmark by saving the succession of system inputs, which, in conjunction with the system characteristics and system parameters, achieve the desired viewing configuration. Thereby, the disclosed system and method permits review of the inputs used to manipulate the image to confirm the process by which a diagnosis or anatomical measurement was made, permits the replay of a sequence for presentation or training purposes to effectively demonstrate not only the end result, but the methodology as well. The disclosed system and method further permits the user to save the system parameters as set for a desired given viewing configuration of a 4D image sequence in addition to or in lieu of saving the system inputs. In this way, the desired viewing configuration is preserved while maintaining the integrity of the underlying image data. In addition, the disclosed system and method further permit the user to maintain or terminate the association of the bookmark 200 with the entirety of the source image data, or just a portion thereof.

Unsupervised discovery of invariances

The grey level profiles of adjacent image regions tend to be different, whilst the ‘hidden’ physical parameters associated with these regions (e.g. surface depth, edge orientation) tend to have similar values. We demonstrate that a network in which adjacent units receive inputs from adjacent image regions learns to code for hidden parameters. The learning rule takes advantage of the spatial smoothness of physical parameters in general to discover particular parameters embedded in grey level profiles which vary rapidly across an input image. We provide examples in which networks discover stereo disparity and feature orientation as invariances underlying image data.