High-resolution 3D Models Research Articles

Three-Dimensional Modeling and Extended Reality Simulations of the Cross-Sectional Anatomy of the Cerebrum, Cerebellum, and Brainstem.

Understanding the anatomy of the human cerebrum, cerebellum, and brainstem and their 3-dimensional (3D) relationships is critical for neurosurgery. Although 3D photogrammetric models of cadaver brains and 2-dimensional images of postmortem brain slices are available, neurosurgeons lack free access to 3D models of cross-sectional anatomy of the cerebrum, cerebellum, and brainstem that can be simulated in both augmented reality (AR) and virtual reality (VR). To create 3D models and AR/VR simulations from 2-dimensional images of cross-sectionally dissected cadaveric specimens of the cerebrum, cerebellum, and brainstem. The Klingler method was used to prepare 3 cadaveric specimens for dissection in the axial, sagittal, and coronal planes. A series of 3D models and AR/VR simulations were then created using 360° photogrammetry. High-resolution 3D models of cross-sectional anatomy of the cerebrum, cerebellum, and brainstem were obtained and used in creating AR/VR simulations. Eleven axial, 9 sagittal, and 7 coronal 3D models were created. The sections were planned to show important deep anatomic structures. These models can be freely rotated, projected onto any surface, viewed from all angles, and examined at various magnifications. To our knowledge, this detailed study is the first to combine up-to-date technologies (photogrammetry, AR, and VR) for high-resolution 3D visualization of the cross-sectional anatomy of the entire human cerebrum, cerebellum, and brainstem. The resulting 3D images are freely available for use by medical professionals and students for better comprehension of the 3D relationship of the deep and superficial brain anatomy.

A new shape model of the bilobate comet 67P/Churyumov-Gerasimenko

In this article, we performed the stereo-photogrammetric processing on over 5200 Rosetta/OSIRIS-NAC images to derive a global high-resolution 3D model of the bilobate comet 67P/Churyumov-Gerasimenko, which is visually richer and more convincing than the existing models (i.e., SPG-SHAP4S, SPC-SHAP5 and SPG-SHAP7). This shape model was used to generate new measurements of the volume (18.62 ± 0.02 km3), the mean density (536.1 ± 0.7 kg/m3), and the surface area (54.0 ± 0.1 km2) of the 67P/C-G comet. Moreover, we also analyzed the reasons for the differences between the three models, namely the coverage of the image set and shadows.

Diagnostic Process of an Ancient Colonnade Using 3D High-Resolution Models with Non-Invasive Multi Techniques.

Here, an avant-garde study of three ancient Doric columns of the precious, ancient Romanesque church of Saints Lorenzo and Pancrazio in the historical town center of Cagliari (Italy) is presented based on the integrated application of different non-destructive testing methods. The limitations of each methodology are overcome by the synergistic application of these methods, affording an accurate, complete 3D image of the studied elements. Our procedure begins with a macroscopic in situ analysis to provide a preliminary diagnosis of the conditions of the building materials. The next step is laboratory tests, in which the porosity and other textural characteristics of the carbonate building materials are studied by optical and scanning electron microscopy. After this, a survey with a terrestrial laser scanner and close-range photogrammetry is planned and executed to produce accurate high-resolution 3D digital models of the entire church and the ancient columns inside. This was the main objective of this study. The high-resolution 3D models allowed us to identify architectural complications occurring in historical buildings. The 3D reconstruction with the above metric techniques was indispensable for planning and carrying out the 3D ultrasonic tomography, which played an important role in detecting defects, voids, and flaws within the body of the studied columns by analyzing the propagation of the ultrasonic waves. The high-resolution 3D multiparametric models allowed us to obtain an extremely accurate picture of the conservation state of the studied columns in order to locate and characterize both shallow and internal defects in the building materials. This integrated procedure can aid in the control of the spatial and temporal variations in the materials' properties and provides information on the process of deterioration in order to allow adequate restoration solutions to be developed and the structural health of the artefact to be monitored.

Stochastic Geometric Iterative Method for Loop Subdivision Surface Fitting

In this paper, we propose a stochastic geometric iterative method (S-GIM) to approximate the high-resolution 3D models by finite loop subdivision surfaces. Given an input mesh as the fitting target, the initial control mesh is generated using the mesh simplification algorithm. Then, our method adjusts the control mesh iteratively to make its finite loop subdivision surface approximate the input mesh. In each geometric iteration, we randomly select part of points on the subdivision surface to calculate the difference vectors and distribute the vectors to the control points. Finally, the control points are updated by adding the weighted average of these difference vectors. We prove the convergence of S-GIM and verify it by demonstrating error curves in the experiment. In addition, compared with existing geometric iterative methods, S-GIM has a shorter running time under the same number of iteration steps.

Multi-Sensor Data Fusion for 3D Reconstruction of Complex Structures: A Case Study on a Real High Formwork Project

As the most comprehensive document types for the recording and display of real-world information regarding construction projects, 3D realistic models are capable of recording and displaying simultaneously textures and geometric shapes in the same 3D scene. However, at present, the documentation for much of construction infrastructure faces significant challenges. Based on TLS, GNSS/IMU, mature photogrammetry, a UAV platform, computer vision technologies, and AI algorithms, this study proposes a workflow for 3D modeling of complex structures with multiple-source data. A deep learning LoFTR network was used first for image matching, which can improve matching accuracy. Then, a NeuralRecon network was employed to generate a 3D point cloud with global consistency. GNSS information was used to reduce search space in image matching and produce an accurate transformation matrix between the image scene and the global reference system. In addition, to enhance the effectiveness and efficiency of the co-registration of the two-source point clouds, an RPM-net was used. The proposed workflow processed the 3D laser point cloud and UAV low-altitude multi-view image data to generate a complete, accurate, high-resolution, and detailed 3D model. Experimental validation on a real high formwork project was carried out, and the result indicates that the generated 3D model has satisfactory accuracy with a registration error value of 5 cm. Model comparison between the TLS, image-based, data fusion 1 (using the common method), and data fusion 2 (using the proposed method) models were conducted in terms of completeness, geometrical accuracy, texture appearance, and appeal to professionals. The results denote that the generated 3D model has similar accuracy to the TLS model yet also provides a complete model with a photorealistic appearance that most professionals chose as their favorite.

Mandible segmentation from CT data for virtual surgical planning using an augmented two-stepped convolutional neural network

PurposeFor computer-aided planning of facial bony surgery, the creation of high-resolution 3D-models of the bones by segmenting volume imaging data is a labor-intensive step, especially as metal dental inlays or implants cause severe artifacts that reduce the quality of the computer-tomographic imaging data. This study provides a method to segment accurate, artifact-free 3D surface models of mandibles from CT data using convolutional neural networks.MethodsThe presented approach cascades two independently trained 3D-U-Nets to perform accurate segmentations of the mandible bone from full resolution CT images. The networks are trained in different settings using three different loss functions and a data augmentation pipeline. Training and evaluation datasets consist of manually segmented CT images from 307 dentate and edentulous individuals, partly with heavy imaging artifacts. The accuracy of the models is measured using overlap-based, surface-based and anatomical-curvature-based metrics.ResultsOur approach produces high-resolution segmentations of the mandibles, coping with severe imaging artifacts in the CT imaging data. The use of the two-stepped approach yields highly significant improvements to the prediction accuracies. The best models achieve a Dice coefficient of 94.824% and an average surface distance of 0.31 mm on our test dataset.ConclusionThe use of two cascaded U-Net allows high-resolution predictions for small regions of interest in the imaging data. The proposed method is fast and allows a user-independent image segmentation, producing objective and repeatable results that can be used in automated surgical planning procedures.

Virtual reality learning system for remote rock mass mapping

Rock mass quality mapping is essential in rock engineering and mining projects. However, the current manual mapping approach and its teaching are restricted due to the dangerous nature of work near rock walls and the constraints of resources. Therefore, a virtual reality learning system for teaching remote mapping of rock mass quality was developed at Aalto University. Two rock wall sections of an underground tunnel and a roadside rock-cut were scanned using photogrammetry and imported as high-resolution textured 3D models. The user wears a head-mounted display and performs mapping using a pointing device. The system collects the measurements automatically and assists the user in rock mass mapping. Along with the development of the system, qualitative research was conducted to evaluate user performance in a gamified learning system. The results demonstrate that with proper game elements implemented, the learning system can be used to steer student behaviour towards the target range. Once set up, the VR system is a tireless, user-paced aid in training and teaching.

Identification of homologous GluN subunits variants accelerates GRIN variants stratification.

The clinical spectrum of GRIN-related neurodevelopmental disorders (GRD) results from gene- and variant-dependent primary alterations of the NMDA receptor, disturbing glutamatergic neurotransmission. Despite GRIN gene variants' functional annotations being dually critical for stratification and precision medicine design, genetically diagnosed pathogenic GRIN variants currently outnumber their relative functional annotations. Based on high-resolution crystal 3D models and topological domains conservation between GluN1, GluN2A, and GluN2B subunits of the NMDAR, we have generated GluN1-GluN2A-GluN2B subunits structural superimposition model to find equivalent positions between GluN subunits. We have developed a GRIN structural algorithm that predicts functional changes in the equivalent structural positions in other GluN subunits. GRIN structural algorithm was computationally evaluated to the full GRIN missense variants repertoire, consisting of 4,525 variants. The analysis of this structure-based model revealed an absolute predictive power for GluN1, GluN2A, and GluN2B subunits, both in terms of pathogenicity-association (benign vs. pathogenic variants) and functional impact (loss-of-function, benign, gain-of-function). Further, we validated this computational algorithm experimentally, using an in silico library of GluN2B-equivalent GluN2A artificial variants, designed from pathogenic GluN2B variants. Thus, the implementation of the GRIN structural algorithm allows to computationally predict the pathogenicity and functional annotations of GRIN variants, resulting in the duplication of pathogenic GRIN variants assignment, reduction by 30% of GRIN variants with uncertain significance, and increase by 70% of functionally annotated GRIN variants. Finally, GRIN structural algorithm has been implemented into GRIN variants Database (http://lmc.uab.es/grindb), providing a computational tool that accelerates GRIN missense variants stratification, contributing to clinical therapeutic decisions for this neurodevelopmental disorder.

A novel whole‐brain imaging pipeline for high resolution spatial and temporal measurements of amyloid‐beta plaque dynamics in pre‐clinical Alzheimer’s Disease animal models

AbstractBackgroundAlzheimer’s Disease (AD) has a strong spatial‐temporal component to its progression, where different brain regions are affected by amyloid‐beta (Aβ) plaque deposition at varying time points. Standard imaging and analysis platforms can neglect these details, as they lack the ability to pair high‐yield whole‐brain imaging with region‐specific quantitation. Furthermore, many Aβ analyses require homogenization of tissue, prohibiting secondary analysis. To address this gap, we have developed a novel high‐throughput whole‐brain imaging pipeline for pre‐clinical AD models to quantitatively track plaque progression as a function of brain region across time while producing indexed tissue sections for secondary staining and analysis.MethodAβ plaques in the well characterized 5XFAD mouse model of AD were labeled with Methoxy‐X04, an Aβ plaque‐specific compound, prior to transcardial perfusion and whole‐brain excision. The brains were processed on the TissueCyte Serial Two‐Photon Plus (STP2) imaging platform to produce fully aligned multi‐channel volumetric datasets, yielding high resolution 3D models of each brain. Registration to the Allen Mouse Brain Common Coordinate Framework (CCFv3) and region‐specific plaque analysis were conducted to determine plaque size, density, and total number per animal brain. The regional density of dystrophic neurites (Lamp1) and microglia (Iba1) were segmented and quantified through secondary analysis to evaluate the signal in high and low Aβ growth rate regions for select sections and remapped to the original 3D whole‐brain dataset.ResultThe analysis of whole‐brain plaque distribution in the 5XFAD mouse model revealed distinct spatial‐temporal changes across the brains (Figure 1, Figure 2). STP2 imaging, combined with CCFv3 mapping and secondary analysis of Lamp1 and Iba1 allowed for targeted evaluation of Aβ plaques, and a comparison of regional molecular changes between models.ConclusionOur novel technology has great promise for quantifying the spatial‐temporal Aβ plaque efficacy of AD animal models, and the production of translatable pre‐clinical AD drug discovery data. The high sensitivity and precision of the STP2 platform can benefit region‐specific disease progression compared to standard laboratory approaches.

Ultraliser: a framework for creating multiscale, high-fidelity and geometrically realistic 3D models for in silico neuroscience.

There is crystal clear evidence on the impact of cell shape on its signaling mechanisms. Structural models can therefore be insightful to realize the function; the more realistic the structure can be, the further we get insights into the function. Creating realistic structural models from existing ones is challenging, particularly when needed for detailed subcellular simulations. We present Ultraliser, a neuroscience-dedicated framework capable of building these structural models with realistic and detailed cellular geometries that can be used for simulations.

Prior depth-based multi-view stereo network for online 3D model reconstruction

This study addresses the online multi-view stereo (MVS) problem when reconstructing precise 3D models in real time. To solve this problem, most previous studies adopted a motion stereo approach that sequentially estimates depth maps from multiple localized images captured in a local time window. To compute the depth maps quickly, the motion stereo methods process down-sampled images or use a simplified algorithm for cost volume regularization; therefore, they generally produce reconstructed 3D models that are inaccurate. In this paper, we propose a novel online MVS method that accurately reconstructs high-resolution 3D models. This method infers prior depth information based on sequentially estimated depths and leverages it to estimate depth maps more precisely. The method constructs a cost volume by using the prior-depth-based visibility information and then fuses the prior depths into the cost volume. This approach significantly improves the stereo matching performance and completeness of the estimated depths. Extensive experiments showed that the proposed method outperforms other state-of-the-art MVS and motion stereo methods. In particular, it significantly improves the completeness of 3D models.

Gli Archivi Digitali e l'Antropologia Virtuale. Ricostruzione 3D di un cranio umano mediante la moderna tecnica della Fotogrammetria

Modern digital archives mainly thanks to recent developments in 3D modelling by TAC, Laser Scanner and Photogrammetry are highy useful for easily storing and exchanging large amounts of data for both qualitative and quantitative studies. Photogrammetry is a versatile and inexpensive technique that starting from one or more sets of 2D digital images allows for high-resolution 3D models to be built. The protocol reported here allows users with different backgrounds to easily and quickly built accurate models of human skulls using minimal and inexpensive equipment by simply modifying only some of the default settings of the Agisoft-Metashape software. The photo acquisition and the workflow of the software (fundamental steps of photogrammetry) are treated separately to provide a systematic method. The main advantages of this method are: the low number of images needed for a total coverage of the skull, the processing speed and the model accuracy by using only four markers positioned on the skull, as a reference to join the two sets of photos.

A Note on Macroscopic Optical Coherence Tomography Imaging Enabled 3D Scanning for Museum and Cultural Heritage Applications

ABSTRACT Optical coherence tomography (OCT) is a non-invasive imaging method that can be used to study the surface features and subsurface structures of delicate cultural heritage objects. However, the field of view of OCT severely limits the system’s scanning area. Previously, we have presented a hybrid scanning platform combined with an imaging stitching algorithm to achieve macroscopic OCT (macro-OCT) imaging. This paper further demonstrates the potential applications of the OCT data by rendering 3D volumetric data into standard virtual reality (VR), augmented reality (AR), and 3D printing formats. The 3D model can be 3D printed or interactively displayed through various platforms such as VR and AR headsets, smartphones, and web pages. The high-resolution 3D models obtained from the macro-OCT system can potentially improve the experience of accessing artworks online and assist individuals with visual impairments to study art through tactile feedback.

Three-Dimensional Modeling and Augmented and Virtual Reality Simulations of the White Matter Anatomy of the Cerebrum.

An understanding of the anatomy of white matter tracts and their 3-dimensional (3D) relationship with each other is important for neurosurgical practice. The fiber dissection technique contributes to this understanding because it involves removing the brain's white matter tracts to reveal their anatomic organization. Using this technique, we built freely accessible 3D models and augmented and virtual reality simulations of white matter tracts. To define the white matter tracts of cadaveric human brains through fiber dissection and to make 2-dimensional and 3D images of the white matter tracts and create 3D models and augmented and virtual reality simulations. Twenty cadaveric brain specimens were prepared in accordance with the Klingler method. Brain hemispheres were dissected step-by-step from lateral-to-medial and medial-to-lateral directions. Three-dimensional models and augmented reality and virtual reality simulations were built with photogrammetry. High-resolution 3D models and augmented reality and virtual reality simulations of the white matter anatomy of the cerebrum were obtained. These models can be freely shifted and rotated on different planes, projected on any real surface, visualized from both front and back, and viewed from various angles at various magnifications. To our knowledge, this is the first detailed study integrating various technologies (3D modeling, augmented reality, and virtual reality) for high-resolution 3D visualization of dissected white matter fibers of the entire human cerebrum.

Photogrammetric reconstruction of 3D carpological collection in high resolution for plants authentication and species discovery.

This study provides an accurate and efficient method to reconstruct detailed and high-resolution digital 3D models of carpological materials by photogrammetric method, in which only about 100 to 150 images are required for each model reconstruction. The 3D models reflect the realistic morphology and genuine color of the carpological materials. The 3D models are scaled to represent the true size of the materials even as small as 3 mm in diameter. The interfaces are interactive, in which the 3D models can be rotated in 360° to observe the structures and be zoomed to inspect the macroscopic details. This new platform is beneficial for developing a virtual herbarium of carpological collection which is thus the most important to botanical authentication and education.

Intensifying the spatial resolution of 3D thermal models from aerial imagery using deep learning-based image super-resolution

Nowadays, 3D thermal models can play an important role in buildings' energy management while acquiring multisource data to generate a high-resolution 3D thermal model. Consequently, in this article, a method for intensifying 3D thermal model using deep learning-based image super-resolution is presented. In the proposed method, first, the enhanced deep residual super-resolution (EDSR) deep network is re-trained based on thermal aerial images. Second, the resolution of low-resolution thermal images is enhanced using the newly trained network. Finally, the state-of-the-art structures from motion (SfM), semi global matching (SGM) and space intersection are utilized to generate intensified 3D thermal model from the resolution enhanced thermal images. Spatial evaluations indicate a 5% increase in edge-based image fusion metric (EFM) for the intensified 3D model. Besides, the evaluations show that the modulation transfer function (MTF) curves of the intensified 3D thermal model are closer to a reference model against the original 3D thermal model. Highlights A 3D thermal model intensification solution using EDSR is proposed which is independent of hardware techniques and multisource data. Considering the importance of edge sharpness in the intensified 3D thermal model, the quality of edges is assessed using MTF curves and the EFM metric. In comparison to the original 3D thermal model, the MTF curves of the intensified 3D thermal model are closer to the MTF curve of the high-resolution 3D model. The EFM metric shows higher values for MTF curves of the intensified 3D thermal model against MTF curves of the original 3D thermal model.

VERY HIGH-RESOLUTION 3D SURVEYING AND MODELLING EXPERIENCES IN CIVIL ENGINEERING APPLICATIONS

Abstract. In this paper some experiences in 3D modelling of objects with very high-resolution are described, carried out by the DICAM Geomatics group of the University of Bologna in multi-disciplinary contexts within the field of the Civil Engineering. In all the addressed case studies the main aim is the generation of a 3D model of the surface at a sub-millimetric scale, allowing a very accurate characterization of the surface geometry, useful for different purposes. 3D scanning and Structure from Motion photogrammetry have been used to generate the 3D models. In the paper the encountered problems and the adopted solutions in data surveying and processing are underlined, also discussing the added value of very high-resolution 3D modelling in multi-disciplinary activities.

Terrestrial laser scanning for the modelling of indoor archaeological areas: two case-studies from Reggio Calabria, Italy

Abstract The work shows the application of a portable terrestrial laser scanner for the three-dimensional survey and modelling of two archaeological areas preserved in the urban fabric of Reggio Calabria, Italy: the underground site of Piazza Italia, better known as “Ipogeo”, and the ruins of the s.c. odéon, located in the basement of a residential building in via 24 Maggio. The two case studies, compared to the other sites preserved in the city center, share a common element: they are not outdoors but have been incorporated into modern buildings and therefore they are not immediately visible. Due to the specific features of the sites, the use of a portable laser-scanning system has been considered the most suitable, accurate and quick solution in order to produce high-resolution as-built 3d models of the structures. The latter will be used as support to derive plans and sections, to carry out further virtual reconstructions and to plan a scan-to-bim process.

Seismic behaviour and 3D reconstruction of Cultural Heritage: the Column of the Hera Lacinia Temple in Capo Colonna (Calabria, Southern Italy)

Abstract The study aims to characterize the seismic behaviour of the Doric Column, last remaining of the Hera Lacinia Temple, located on the promontory of Capo Colonna near Crotone (Calabria, Southern Italy). Recordings of ambient vibration has been acquired both at the base and on the top of the column, simultaneously, in order to detect the predominant frequencies of the element. In addition, a high-resolution 3D model of the column has been reconstructed through laser scanning to support the analysis of the current state of the heritage element. The fundamental frequencies of the column and those associated to the site have been compared to verify potential resonance phenomena at the occurrence of an earthquake. In addition, a significant inclination of the column was detected by analysing the 3D model.

An out-of-core method for GPU image mapping on large 3D scenarios of the real world

Image mapping on 3D huge scenarios of the real world is one of the most fundamental and computational expensive processes for the integration of multi-source sensing data. Recent studies focused on the observation and characterization of Earth have been enhanced by the proliferation of Unmanned Aerial Vehicle (UAV) and sensors able to capture massive datasets with a high spatial resolution. Despite the advances in manufacturing new cameras and versatile platforms, only a few methods have been developed to characterize the study area by fusing heterogeneous data such as thermal, multispectral or hyperspectral images with high-resolution 3D models. The main reason for this lack of solutions is the challenge to integrate multi-scale datasets and high computational efforts required for image mapping on dense and complex geometric models. In this paper, we propose an efficient pipeline for multi-source image mapping on huge 3D scenarios. Our GPU-based solution significantly reduces the run time and allows us to generate enriched 3D models on-site. The proposed method is out-of-core and it uses available resources of the GPU’s machine to perform two main tasks: (i) image mapping and (ii) occlusion testing. We deploy highly-optimized GPU-kernels for image mapping and detection of self-hidden geometry in the 3D model, as well as a GPU-based parallelization to manage the 3D model considering several spatial partitions according to the GPU capabilities. Our method has been tested on 3D scenarios with different point cloud densities (66M, 271M, 542M) and two sets of multispectral images collected by two drone flights. We focus on launching the proposed method on three platforms: (i) System on a Chip (SoC), (ii) a user-grade laptop and (iii) a PC. The results demonstrate the method’s capabilities in terms of performance and versatility to be computed by commodity hardware. Thus, taking advantage of GPUs, this method opens the door for embedded and edge computing devices for 3D image mapping on large-scale scenarios in near real-time.