Resolution 3D Data Research Articles

Three-Dimensional-Slice-Super-Resolution-Net: A Fast Few Shooting Learning Model for 3D Super-Resolution Using Slice-Up and Slice-Reconstruction

Abstract 3D modeling accurately depicts the physical world but typically requires substantial data acquisition resources and significant storage space. We introduce a novel three-dimensional slice-reconstruction model (3DSR) to address these challenges. This 3D data super-resolution model leverages low-resolution 3D data as input to generate high-resolution results promptly and accurately, reducing the time and storage required to create detailed 3D models. To enhance the computational efficiency and accuracy of deep learning models, the 3D data are partitioned into multiple slices. The 3DSR processes each slice into a high-resolution 2D image, which is then reassembled into high-resolution 3D data. Our slice-up method and slice-reconstruction technique are specifically designed to preserve the primary characteristics of the 3D data. We employ a pre-trained deep 2D convolutional neural network to expand the resolution of the 2D image, resulting in excellent performance. This approach reduces the time required for training deep learning models and enhances computational efficiency during the resolution improvement process. Importantly, our model can deliver superior performance even when trained on fewer data.

A Persistent Homology-Based Topological Loss for CNN-Based Multiclass Segmentation of CMR.

Multi-class segmentation of cardiac magnetic resonance (CMR) images seeks a separation of data into anatomical components with known structure and configuration. The most popular CNN-based methods are optimised using pixel wise loss functions, ignorant of the spatially extended features that characterise anatomy. Therefore, whilst sharing a high spatial overlap with the ground truth, inferred CNN-based segmentations can lack coherence, including spurious connected components, holes and voids. Such results are implausible, violating anticipated anatomical topology. In response, (single-class) persistent homology-based loss functions have been proposed to capture global anatomical features. Our work extends these approaches to the task of multi-class segmentation. Building an enriched topological description of all class labels and class label pairs, our loss functions make predictable and statistically significant improvements in segmentation topology using a CNN-based post-processing framework. We also present (and make available) a highly efficient implementation based on cubical complexes and parallel execution, enabling practical application within high resolution 3D data for the first time. We demonstrate our approach on 2D short axis and 3D whole heart CMR segmentation, advancing a detailed and faithful analysis of performance on two publicly available datasets.

Experimental Analysis of Geo-spatial Data to Evaluate Urban Greenspace: A Case Study in Dortmund, Germany

Urban greenspace is of high importance for the general public from different points of view. Among others, it is beneficial for recreational activities as well as for cooling effects during summer in urban climate. Many studies investigate urban greenspace from perspectives related to environmental justice or health geography that are usually based on environmental and sociodemographic data of diverse scales or spatial levels of detail and often suffer from high degrees of generalization. To demonstrate the benefit of a higher degree of spatial detail, a representative part of the city of Dortmund (Germany) is selected as study area. A time series of freely available Sentinel-2 images and a freely available digital object height model (DOHM) are used to analyze and describe the spatio-temporal distribution of urban vegetation and its potential effects on the population. Doing so, the Sentinel-2 images are used for the calculation of vegetation indices to identify the active green period of different vegetation land covers. In addition to that, the three-dimensional appearance of vegetation is correlated with the green period to assume the relevance of green land covers for the general public. The results show that an appropriate selection of cloud-free satellite images is of importance for a greenspace analysis. The combination of high resolution 2D and 3D data sets, namely NDVI and plant height information, reveals the spatial distribution of well-equipped greenspace areas with high recreational and health potential as well as poorly-equipped areas in urban neighborhoods, that could be improved.

Correlating 3D porous structure in polymer films with mass transport properties using FIB-SEM tomography

Porous polymer coatings are used to control drug release from pharmaceutical products. The coating covers a drug core and depending on the porous structure, different drug release rates are obtained. This work presents mass transport simulations performed on porous ethyl cellulose films with different porosities. The simulations were performed on high spatial resolution 3D data obtained using a focused ion beam scanning electron microscope. The effective diffusion coefficient of water was determined using a diffusion chamber. Lattice Boltzmann simulations were used to simulate water diffusion in the 3D data. The simulated coefficient was in good agreement with the measured coefficient. From the results it was concluded that the tortuosity and constrictivity of the porous network increase with decreasing amount of added hydroxypropyl cellulose, resulting in a sharp decrease in effective diffusion. This work shows that high spatial resolution 3D data is necessary, and that 2D data is insufficient, in order to predict diffusion through the porous structure with high accuracy.

3D high spatial resolution visualisation and quantification of interconnectivity in polymer films.

A porous network acts as transport paths for drugs through films for controlled drug release. The interconnectivity of the network strongly influences the transport properties. It is therefore important to quantify the interconnectivity and correlate it to transport properties for control and design of new films. This work presents a novel method for 3D visualisation and analysis of interconnectivity. High spatial resolution 3D data on porous polymer films for controlled drug release has been acquired using a focused ion beam (FIB) combined with a scanning electron microscope (SEM). The data analysis method enables visualisation of pore paths starting at a chosen inlet pore, dividing them into groups by length, enabling a more detailed quantification and visualisation. The method also enables identification of central features of the porous network by quantification of channels where pore paths coincide. The method was applied to FIB-SEM data of three leached ethyl cellulose (EC)/hydroxypropyl cellulose (HPC) films with different weight percentages. The results from the analysis were consistent with the experimentally measured release properties of the films. The interconnectivity and porosity increase with increasing amount of HPC. The bottleneck effect was strong in the leached film with lowest porosity.

ModLayer: A MATLAB GUI Drawing Segmentation Tool for Visualizing and Classifying 3D Data

Characterizing a material’s microstructure, especially as it relates to the manufacturing processes used to fabricate it, is of great interest to engineers and researchers. In recent years, state-of-the-art imaging techniques have been able to yield a plethora of high resolution 3D data that can be used to study materials at various length scales. This 3D data is usually organized as stacked serial sections of 2D images and almost always requires some combination of enhancement and segmentation (the process of separating an image into subsets), in order to extract meaningful information. To aid in this process, ModLayer was created as a MATLAB® executable. ModLayer is an interactive graphical user interface that seeks to remove the burden of import/export redundancies when interacting with 3D data in MATLAB during visualization, modification, or segmentation through manual drawing across image stacks. The utility of ModLayer is demonstrated here through three case studies; (1) classifying regions of damage with in-situ time lapse X-ray micro-computed tomography ($$\mu $$-CT) of a glass fiber reinforced polypropylene (GFRP), (2) correcting multi-class segmentation errors in segmented X-ray $$\mu $$-CT images of a GRFP composite, and (3) capturing features of interest within in-situ 3D X-ray $$\mu $$-CT images during fatigue crack growth experiments of aluminum 7050. Overall, this tool is especially useful to engineers and researchers interested in correcting—within MATLAB—automated segmentation of noisy 3D images which can yield erroneous microstructural features in segmentation procedures.

Dasymetric Mapping Using UAV High Resolution 3D Data within Urban Areas

Multi-temporal analysis of census small-area microdata is hampered by the fact that census tract shapes do not often coincide between census exercises. Dasymetric mapping techniques provide a workaround that is nonetheless highly dependent on the quality of ancillary data. The objectives of this work are to: (1) Compare the use of three spatial techniques for the estimation of population according to census tracts: Areal interpolation and dasymetric mapping using control data—building block area (2D) and volume (3D); (2) demonstrate the potential of unmanned aerial vehicle (UAV) technology for the acquisition of control data; (3) perform a sensitivity analysis using Monte Carlo simulations showing the effect of changes in building block volume (3D information) in population estimates. The control data were extracted by a (semi)-automatic solution—3DEBP (3D extraction building parameters) developed using free open source software (FOSS) tools. The results highlight the relevance of 3D for the dasymetric mapping exercise, especially if the variations in height between building blocks are significant. Using low-cost UAV backed systems with a FOSS-only computing framework also proved to be a competent solution with a large scope of potential applications.

Apple orchard inventory with a LiDAR equipped unmanned aerial system

Knowledge about the number of trees in an orchard and their geometric parameters is beneficial in precise farming and together with other information may be used to predict the yield. These parameters can be obtained based on time-consuming field measurements or more effectively, from very high resolution 3D data collected with Unmanned Aerial Vehicles (UAV). Numerous UAV experiments have been conducted in agricultural areas; however, most of studies are limited to the use of a passive optical sensor (camera). This study demonstrates an experiment on the novel remote sensing approach of determining selected geometric parameters of trees in an apple orchard, based on a high-density point cloud obtained from a Velodyne HDL-32E laser scanner mounted on a small UAV platform Leica Aibot X6 V2. Reference data of selected geometric parameters of trees was obtained from orthophotomap and with geodetic surveying methods. Original and robust methodology is proposed for the point cloud processing, which is the inventive combination of an alpha-shape algorithm, principal component analysis and detection of local minima on crown profiles. The developed approach allowed for the correct identification of 99% of the trees in the test orchard. The root mean square error of determined crown areas was equal to 0.98 m2. The accuracy of tree top identification, tree height and crown base height determination was equal to 0.38, 0.09 and 0.09 m, respectively.

Rapid T1 quantification from high resolution 3D data with model‐based reconstruction

PurposeMagnetic resonance imaging protocols for the assessment of quantitative information suffer from long acquisition times since multiple measurements in a parametric dimension are required. To facilitate the clinical applicability, accelerating the acquisition is of high importance. To this end, we propose a model‐based optimization framework in conjunction with undersampling 3D radial stack‐of‐stars data.Theory and MethodsHigh resolution 3D T 1 maps are generated from subsampled data by employing model‐based reconstruction combined with a regularization functional, coupling information from the spatial and parametric dimension, to exploit redundancies in the acquired parameter encodings and across parameter maps. To cope with the resulting non‐linear, non‐differentiable optimization problem, we propose a solution strategy based on the iteratively regularized Gauss‐Newton method. The importance of 3D‐spectral regularization is demonstrated by a comparison to 2D‐spectral regularized results. The algorithm is validated for the variable flip angle (VFA) and inversion recovery Look‐Locker (IRLL) method on numerical simulated data, MRI phantoms, and in vivo data.ResultsEvaluation of the proposed method using numerical simulations and phantom scans shows excellent quantitative agreement and image quality. T 1 maps from accelerated 3D in vivo measurements, e.g. 1.8 s/slice with the VFA method, are in high accordance with fully sampled reference reconstructions.ConclusionsThe proposed algorithm is able to recover T 1 maps with an isotropic resolution of 1 mm3 from highly undersampled radial data by exploiting structural similarities in the imaging volume and across parameter maps.

Internal Structural Imaging of Cultural Wooden Relics Based on Three-Dimensional Computed Tomography

An efficient method for the internal structural imaging of cultural wooden relics was explored through experimental techniques of three-dimensional (3-D) tomography and reconstruction. The techniques of filtering and segmentation were applied to the 3-D scanned data of wooden cultural relics.To obtain high resolution 3D data model, it was necessary to preprocess the raw data after CT scanning. Preprocessing included denoising, filtering, and segmentation. After completing these three steps, three-dimensional reconstruction experiments were carried out (including surface rendering and volume rendering). After the 3-D reconstruction, the wood internal properties were visually analyzed and used to create internal structural imaging of wooden artifacts. On the basis of volume rendering, wooden artifacts could be graphically divided at any angle and any position. The textures of local wooden relics were clearly revealed in the segmentation of the reconstruction pictures, and these were compared with the presented internal structural image testing of the wooden artifacts. This study showed that the proposed technology can successfully create internal structural images of wooden artifacts, as well as provide important data and models to support the renovation and recovery of the cultural wooden relics.

Combining Coherent Hard X-Ray Tomographies with Phase Retrieval to Generate Three-Dimensional Models of Forming Bone

Holotomography, a phase sensitive synchrotron-based μCT modality, is a quantitative 3D imaging method. By exploiting partial spatial X-ray coherence, bones can be imaged volumetrically with high resolution coupled with impressive density sensitivity. This tomographic method reveals the main characteristics of the important tissue compartments in forming bones, including the rapidly-changing soft tissue and the partially or fully mineralized bone regions, while revealing subtle density differences in 3D. Here we show typical results observed within the growing femur bone midshafts of healthy mice that are 1, 3, 7, 10 and 14 days old (postpartum). Our results make use of partially-coherent synchrotron radiation employing inline Fresnel-propagation in multiple tomographic datasets obtained in the imaging beamline ID19 of the ESRF. The exquisite detail creates maps of the juxtaposed soft, partially mineralized and highly mineralized bone revealing the environment in which bone cells create and shape the matrix. This high resolution 3D data is a step towards creating realistic computational models that may be used to study the dynamic processes involved in bone tissue formation and adaptation. Such data will enhance our understanding of the important biomechanical interactions directing maturation and shaping of the bone micro- and macro-geometries.

RANDOM-FOREST-ENSEMBLE-BASED CLASSIFICATION OF HIGH-RESOLUTION REMOTE SENSING IMAGES AND NDSM OVER URBAN AREAS

Abstract. As an intermediate step between raw remote sensing data and digital urban maps, remote sensing data classification has been a challenging and long-standing research problem in the community of remote sensing. In this work, an effective classification method is proposed for classifying high-resolution remote sensing data over urban areas. Starting from high resolution multi-spectral images and 3D geometry data, our method proceeds in three main stages: feature extraction, classification, and classified result refinement. First, we extract color, vegetation index and texture features from the multi-spectral image and compute the height, elevation texture and differential morphological profile (DMP) features from the 3D geometry data. Then in the classification stage, multiple random forest (RF) classifiers are trained separately, then combined to form a RF ensemble to estimate each sample’s category probabilities. Finally the probabilities along with the feature importance indicator outputted by RF ensemble are used to construct a fully connected conditional random field (FCCRF) graph model, by which the classification results are refined through mean-field based statistical inference. Experiments on the ISPRS Semantic Labeling Contest dataset show that our proposed 3-stage method achieves 86.9% overall accuracy on the test data.

Impacts of Sample Size on Calculation of Pavement Texture Indicators with 1mm 3D Surface Data

The emerging 1mm resolution 3D data collection technology is capable of covering the entire pavement surface, and provides more data sets than traditional line-of-sight data collection systems. As a result, quantifying the impact of sample size including sample width and sample length on the calculation of pavement texture indicators is becoming possible. In this study, 1mm 3D texture data are collected and processed at seven test sites using the PaveVision3D Ultra system. Analysis of Variance (ANOVA) test and linear regression models are developed to investigate various sample length and width on the calculation of three widely used texture indicators: Mean Profile Depth (MPD), Mean Texture Depth (MTD) and Power Spectra Density (PSD). Since the current ASTM standards and other procedures cannot be directly applied to 3D surface for production due to a lack of definitions, the results from this research are beneficial in the process to standardize texture indicators’ computations with 1mm 3D surface data of pavements.

Nanoscale Imaging of Collagen Gels with Focused Ion Beam Milling and Scanning Electron Microscopy

In vitro polymerized type I collagen hydrogels have been used extensively as a model system for three-dimensional (3D) cell and tissue culture, studies of fibrillogenesis, and investigation of multiscale force transmission within connective tissues. The nanoscale organization of collagen fibrils plays an essential role in the mechanics of these gels and emergent cellular behavior in culture, yet quantifying 3D structure with nanoscale resolution to fully characterize fibril organization remains a significant technical challenge. In this study, we demonstrate that a new imaging modality, focused ion beam scanning electron microscopy (FIB-SEM), can be used to generate 3D image datasets for visualizing and quantifying complex nanoscale organization and morphometry in collagen gels. We polymerized gels at a number of concentrations and conditions commonly used for in vitro models, stained and embedded the samples, and performed FIB-SEM imaging. The resulting image data had a voxel size of 25 nm, which is the highest resolution 3D data of a collagen fibril network ever obtained for collagen gels. This resolution was essential for discerning individual fibrils, fibril paths, and their branching and grouping. The resulting volumetric images revealed that polymerization conditions have a significant impact on the complex fibril morphology of the gels. We segmented the fibril network and demonstrated that individual collagen fibrils can be tracked in 3D space, providing quantitative analysis of network descriptors such as fibril diameter distribution, length, branch points, and fibril aggregations. FIB-SEM 3D reconstructions showed considerably less lateral grouping and overlap of fibrils than standard 2D SEM images, likely due to artifacts in SEM introduced by dehydration. This study demonstrates the utility of FIB-SEM for 3D imaging of collagen gels and quantitative analysis of 3D fibril networks. We anticipate that the method will see application in future studies of structure-function relationships in collagen gels as well as native collagenous tissues.

Commentary on ‘Orthogonal Rings, Fiducial Markers and Overlay Accuracy When Image Fusion is Used for EVAR Guidance’

Image fusion technology is of value because it has the potential to a useful tool, for optimal c-arm positioning and angulation in particreduce intra-operative radiation and contrast dose. It can provide a 3D roadmap to guide intervention limited, by monoplane systems, to a 2D view in real time. The ideal scenario is as follows: A Cone Beam CT (CBCT) is performed at the beginning of the aortic intervention. This 3D data set is then fused with a higher resolution 3D data set from a conventional thin-slice CTscan (TSCT) and a high quality 3D road map (the fusion image) is produced which obviates or greatly reduces the need for any further contrast use throughout the procedure until completion. Vessel cannulation and device delivery takes place solely based on the Fusion Image (FI), used as a 3D roadmap. Enthusiasm for this technology has been borne out by several publications indicating that, compared with conventional techniques, the application of fusion imaging to both complex and noncomplex aortic interventions is associated with shorter procedure times, less contrast use, and lower radiation dose. Clearly such benefits can be achieved, and not necessarily require precise alignment between the FI and the real time target on fluoroscopy. Koutouzi et al. present an important paper, quantifying the accuracy of 3D CT image fusion for EVAR and indicating the shortfall that exists with current techniques. Firstly examining how the CBCT compares with TSCT in producing the FI and secondly how the FI compares with the DSA image prior to device deployment. Acceptable accuracy was defined as <3 mm misalignment between renal arteries when fusing the TSCTwith theCBCT.Thiswas achieved inonly 37%ofpatients when registration was performed manually using L1 and L2 vertebral alignment, and in none when using an automated process. This suggests that the position of the aorta and the renal arteries relative to the vertebral columnvaries from the TSCT to the intraoperatively CBCT.We also see variation between the CBCT and the pre-deployment DSA image (using renal arteries or fiducial markers as reference points). Only 58% of patients had misalignment of 3 mm or less. Part of this is shownby the authors tobedue topatientmovement intra-operatively, and part of it, is likely to be related to change in the shape of the aorta caused by stiff wires. We can surmise that the position of the renal arteries relative to bony landmarks varies (albeit by millimetres) from one CT to the next and from a CBCT at the start of an EVAR to the pre-deployment angiogram, with stiff wires or an aortic device in place. Current fusion technology cannot yet compensate for this. The authors’ findings are consistent with other recent publications and also emphasise the importance of ongoing manual re-registration. This labour intensive process, required intra-operatively, is an inconvenience as is the potential for the fusion image to impair visualisation of wires or other markers on fluoroscopy. For now, image fusion remains

Network level pavement evaluation with 1 mm 3D survey system

The latest iteration of PaveVision3D Ultra can obtain true 1 mm resolution 3D data at full-lane coverage in all 3 directions at highway speed up to 60 mph. This paper introduces the PaveVision3D Ultra technology for rapid network level pavement survey on approximately 1280 center miles of Oklahoma interstate highways. With sophisticated automated distress analyzer (ADA) software interface, the collected 1 mm 3D data provide Oklahoma Department of Transportation (ODOT) with comprehensive solutions for automated evaluation of pavement surface including longitudinal profile for roughness, transverse profile for rutting, predicted hydroplaning speed for safety analysis, and cracking and various surface defects for distresses. The pruned exact linear time (PELT) method, an optimal partitioning algorithm, is implemented to identify change points and dynamically determine homogeneous segments so as to assist ODOT effectively using the available 1 mm 3D pavement surface condition data for decision-making. The application of 1 mm 3D laser imaging technology for network survey is unprecedented. This innovative technology allows highway agencies to access its options in using the 1 mm 3D system for its design and management purposes, particularly to meet the data needs for pavement management system (PMS), pavement ME design and highway performance monitoring system (HPMS).

Dose accumulation and 3D imaging with He+ ions

Ion implantation accumulation is modeled for the theoretical case of using scanning He+ ion microscopy to image a stack of virtual slices from Si and Cu for 3D tomography. Ion implantation during imaging reaches steady state conditions of maximum concentration at the surface when the virtual thickness of all slices exceeds the total ion range. Imaging at low dose and large voxel sizes offers the greatest likelihood of damage-free targets for the best fidelity 3D data sets. The possibility of damage accumulation increases as the voxel size decreases, which may limit the effectiveness of helium ion microscopy imaging for high resolution 3D data sets. In particular, a surface imaging dose of 2.5 × 1016 ions/cm2 distributed through the volume after each slice can create a situation where almost 2.5 × 1013 ions/cm2 may accumulate at the surface of both Si and Cu at a voxel size of 10 × 10 × 10 nm, possibly causing severe crystalline damage to these targets. This model may be used to predict voxel sizes required for high quality 3D tomography data using helium ion microscopy imaging.

Automatic Single Tree Detection in Plantations using UAV-based Photogrammetric Point clouds

Abstract. For reasons of documentation, management and certification there is a high interest in efficient inventories of palm plantations on the single plant level. Recent developments in unmanned aerial vehicle (UAV) technology facilitate spatial and temporal flexible acquisition of high resolution 3D data. Common single tree detection approaches are based on Very High Resolution (VHR) satellite or Airborne Laser Scanning (ALS) data. However, VHR data is often limited to clouds and does commonly not allow for height measurements. VHR and in particualar ALS data are characterized by high relatively high acquisition costs. Sperlich et al. (2013) already demonstrated the high potential of UAV-based photogrammetric point clouds for single tree detection using pouring algorithms. This approach was adjusted and improved for an application on palm plantation. The 9.4ha test site on Tarawa, Kiribati, comprised densely scattered growing palms, as well as abundant undergrowth and trees. Using a standard consumer grade camera mounted on an octocopter two flight campaigns at 70m and 100m altitude were performed to evaluate the effect Ground Sampling Distance (GSD) and image overlap. To avoid comission errors and improve the terrain interpolation the point clouds were classified based on the geometric characteristics of the classes, i.e. (1) palm, (2) other vegetation (3) and ground. The mapping accuracy amounts for 86.1 % for the entire study area and 98.2 % for dense growing palm stands. We conclude that this flexible and automatic approach has high capabilities for operational use.

Database Creation and Floodplain Mapping of a Residential Layout in Yola, Nigeria

Topographic mapping of flood-prone housing estate has been studied using high resolution 3D data from total station. The total station provides an unprecedented accuracy for a 3D data collection than any other traditional surveying techniques and the satellite-based differential GPS. The Digital Terrain Model (DTM) delineates the potential areas required for drainage construction. It is observed that over 80% of housing facilities could be saved from flood water if proper drainages are constructed as indicated by the DTM. Generally, houses to the east and south would be much affected due to their location in low land areas. Supplementary, a spatial database of the housing estate has been conducted to predict loses that may incur during the perennial and climate-forced floods that affect the area. This database can be queried and based on the entity relationships of each house, potential economic loses can be calculated. It is suggested here that additional options, such as flow direction, slope and watershed analyses could be conducted in future flood mapping of the area.

Remote-sensing techniques for analysing landslide kinematics: a review

Abstract Surface displacement field of landslides is a key parameter to access to their geometries and mechanical properties. Surface displacements can be calculated using remote-sensing methods such as interferometry for radar data and image correlation for optical data. These methods have been elaborated this last decade and successfully applied on sensors (radar, cameras, terrestrial 3D laser scanner imaging) either attached to space or aerial platforms such as satellites, planes, and unmanned radio-controlled platforms (drones and helicopters) or settled at fixed positions emplaced in the front of landslides. This paper reviews the techniques of image analysis (interferometry and optical data correlation) to measure displacements and examines the performance of each type of platforms. Examples of applications of these techniques in French South Alps are shown. Depending on the landslide characteristics (exposure conditions, size, velocity) as well as the goal of the study (operational or scientific purpose), one or a combination of several techniques and data (characterized by several resolution, accuracy, covered surface, revisiting time) have to be used. Radar satellite data processed with differential interferometric or PS methods are mainly suitable for scientific purposes due to various application limitations in mountainous area. Optical satellite and aerial images can be used for scientific studies at fairly high resolution but are strongly dependant on atmospheric conditions. Platforms and sensors such as drone, fixed camera, fixed radar and Lidar have the advantage of high adaptability. They can be used to obtain very high resolution and precise 3D data (of centimetric accuracy) suitable for both scientific and operational purposes.