Large-scale 3D Data Research Articles

Reconstruction of large-scale anisotropic 3D digital rocks from 2D shale images using generative adversarial network

Digital rock technology has played a significant role in unconventional oil and gas exploration and development. Current techniques face challenges such as high cost or low efficiency when reconstructing 3D pore structures at the nanoscale, which limit the study of rock physical properties in reservoirs. To address these issues, this work proposes an efficient generative adversarial network workflow for generating large-scale anisotropic 3D digital rocks from 2D images. Instead of relying on 3D data, the model is trained using only 2D images. During training, rock images taken from various angles can be utilized to construct anisotropic 3D digital rocks. Moreover, a novel method for generating large-scale 3D data is devised, where the input random noise is divided with overlap to generate small-scale digital rocks and concatenated into a larger-scale 3D digital rock. The similarity between the 3D reconstructed data and the training images was assessed separately, as well as the pore structure of the 3D reconstructed data, and the feasibility of the proposed method was verified using the simulation of absolute permeability. The digital rock modeling process presented in this study enables rapid and accurate 2D-to-3D large-scale digital rock modeling, providing an effective representation of the anisotropy present in shale oil reservoirs.

FVDB : A Deep-Learning Framework for Sparse, Large Scale, and High Performance Spatial Intelligence

We present f VDB, a novel GPU-optimized framework for deep learning on large-scale 3D data. f VDB provides a complete set of differentiable primitives to build deep learning architectures for common tasks in 3D learning such as convolution, pooling, attention, ray-tracing, meshing, etc. f VDB simultaneously provides a much larger feature set (primitives and operators) than established frameworks with no loss in efficiency: our operators match or exceed the performance of other frameworks with narrower scope. Furthermore, f VDB can process datasets with much larger footprint and spatial resolution than prior works, while providing a competitive memory footprint on small inputs. To achieve this combination of versatility and performance, f VDB relies on a single novel VDB index grid acceleration structure paired with several key innovations including GPU accelerated sparse grid construction, convolution using tensorcores, fast ray tracing kernels using a Hierarchical Digital Differential Analyzer algorithm (HDDA), and jagged tensors. Our framework is fully integrated with PyTorch enabling interoperability with existing pipelines, and we demonstrate its effectiveness on a number of representative tasks such as large-scale point-cloud segmentation, high resolution 3D generative modeling, unbounded scale Neural Radiance Fields, and large-scale point cloud reconstruction.

Efficient 3D Spatial Queries for Complex Objects.

3D spatial data has been generated at an extreme scale from many emerging applications, such as high definition maps for autonomous driving and 3D Human BioMolecular Atlas. In particular, 3D digital pathology provides a revolutionary approach to map human tissues in 3D, which is highly promising for advancing computer-aided diagnosis and understanding diseases through spatial queries and analysis. However, the exponential increase of data at 3D leads to significant I/O, communication, and computational challenges for 3D spatial queries. The complex structures of 3D objects such as bifurcated vessels make it difficult to effectively support 3D spatial queries with traditional methods. In this article, we present our work on building an efficient and scalable spatial query system, iSPEED, for large-scale 3D data with complex structures. iSPEED adopts effective progressive compression for each 3D object with successive levels of detail. Further, iSPEED exploits structural indexing for complex structured objects in distance-based queries. By querying with data represented in successive levels of details and structural indexes, iSPEED provides an option for users to balance between query efficiency and query accuracy. iSPEED builds in-memory indexes and decompresses data on-demand, which has a minimal memory footprint. iSPEED provides a 3D spatial query engine that can be invoked on-demand to run many instances in parallel implemented with, but not limited to, MapReduce. We evaluate iSPEED with three representative queries: 3D spatial joins, 3D nearest neighbor query, and 3D spatial proximity estimation. The extensive experiments demonstrate that iSPEED significantly improves the performance of existing spatial query systems.

3-D Image-Domain Least-Squares Reverse Time Migration With L1 Norm Constraint and Total Variation Regularization

Data-domain least-squares reverse time migration (DDLSRTM) has been proved to be a more effective imaging tool for complex structures, relative to the standard reverse time migration (RTM) approach. One of the difficulties in DDLSRTM is that the enormous computational costs may impede its application in large-scale 3D data. To mitigate this problem, with the help of point spread functions (PSFs) and spatial interpolation, we have developed a novel 3D image-domain least-squares reverse time migration (IDLSRTM) approach, which requires once migration and modeling calculations. However, because of the incomplete acquisition geometry of seismic recordings, IDLSRTM is a highly ill-posed inverse problem. The inverted image from the conventional IDLSRTM approach may suffer from the migration artifacts caused by the coarse source and receiver sampling and spatial discontinuity and instability caused by the truncated PSFs. To solve the ill-posedness and improve image quality, the L1 norm constraint and total variation (TV) regularization are introduced into the objective function of IDLSRTM. The alternating direction method of multipliers (ADMM) algorithm is developed to solve this optimization problem. Through some 3D synthetic and field data, it can determine that the proposed IDLSRTM approach computationally efficient produces a high-fidelity reflection image with good spatial continuity and fewer migration artifacts. It has shown this approach to be a cost-effective and practical inversion-based imaging tool for 3D field datasets.

Projection-Based Point Convolution for Efficient Point Cloud Segmentation

Understanding point cloud has recently gained huge interests following the development of 3D scanning devices and the accumulation of large-scale 3D data. Most point cloud processing algorithms can be classified as either point-based or voxel-based methods, both of which have severe limitations in processing time or memory, or both. To overcome these limitations, we propose Projection-based Point Convolution (PPConv), a point convolutional module that uses 2D convolutions and multi-layer perceptrons (MLPs) as its components. In PPConv, point features are processed through two branches: point branch and projection branch. Point branch consists of MLPs, while projection branch transforms point features into a 2D feature map and then apply 2D convolutions. As PPConv does not use point-based or voxel-based convolutions, it has advantages in fast point cloud processing. When combined with a learnable projection and effective feature fusion strategy, PPConv achieves superior efficiency compared to state-of-the-art methods, even with a simple architecture based on PointNet++. We demonstrate the efficiency of PPConv in terms of the trade-off between inference time and segmentation performance. The experimental results on S3DIS and ShapeNetPart show that PPConv is the most efficient method among the compared ones. The code is available at github.com/pahn04/PPConv.

Accuracy assessment of RTK-GNSS equipped UAV conducted as-built surveys for construction site modelling

Regular as-built surveys have become a necessary input for building information modelling. Such large-scale 3D data capturing can be conducted effectively by combining structure-from-motion and unmanned aerial vehicles (UAV). Using a RTK-GNSS equipped UAV, 22 repeated weekly campaigns were conducted at two altitudes in various conditions. The photogrammetric approach yielded 3D models, which were compared to the terrestrial laser scanning based ground truth. Better than 2.8 cm geometry RMSE was consistently achieved using integrated georeferencing. It is concluded that the RTK-GNSS based georeferencing enables reaching better than 5 cm geometry accuracy by utilising at least one ground control point.

Can regional aerial images from orthophoto surveys produce high quality photogrammetric Canopy Height Model? A single tree approach in Western Europe

Forest monitoring tools are needed to promote effective and data driven forest management and forest policies. Remote sensing techniques can increase the speed and the cost-efficiency of the forest monitoring as well as large scale mapping of forest attribute (wall-to-wall approach). Digital Aerial Photogrammetry (DAP) is a common cost-effective alternative to airborne laser scanning (ALS) which can be based on aerial photos routinely acquired for general base maps. DAP based on such pre-existing dataset can be a cost effective source of large scale 3D data. In the context of forest characterization, when a quality Digital Terrain Model (DTM) is available, DAP can produce photogrammetric Canopy Height Model (pCHM) which describes the tree canopy height. While this potential seems pretty obvious, few studies have investigated the quality of regional pCHM based on aerial stereo images acquired by standard official aerial surveys. Our study proposes to evaluate the quality of pCHM individual tree height estimates based on raw images acquired following such protocol using a reference filed-measured tree height database. To further ensure the replicability of the approach, the pCHM tree height estimates benchmarking only relied on public forest inventory (FI) information and the photogrammetric protocol was based on low-cost and widely used photogrammetric software. Moreover, our study investigates the relationship between the pCHM tree height estimates based on the neighboring forest parameter provided by the FI program.Our results highlight the good agreement of tree height estimates provided by pCHM using DAP with both field measured and ALS tree height data. In terms of tree height modeling, our pCHM approach reached similar results than the same modeling strategy applied to ALS tree height estimates. Our study also identified some of the drivers of the pCHM tree height estimate error and found forest parameters like tree size (diameter at breast height) and tree type (evergreenness/deciduousness) as well as the terrain topography (slope) to be of higher importance than image survey parameters like the variation of the overlap or the sunlight condition in our dataset. In combination with the pCHM tree height estimate, the terrain slope, the Diameter at Breast Height (DBH) and the evergreenness factor were used to fit a multivariate model predicting the field measured tree height. This model presented better performance than the model linking the pCHM estimates to the field tree height estimates in terms of r² (0.90 VS 0.87) and root mean square error (RMSE, 1.78 VS 2.01 m). Such aspects are poorly addressed in literature and further research should focus on how pCHM approaches could integrate them to improve forest characterization using DAP and pCHM. Our promising results can be used to encourage the use of regional aerial orthophoto surveys archive to produce large scale quality tree height data at very low additional costs, notably in the context of updating national forest inventory programs.

SegMap: Segment-based mapping and localization using data-driven descriptors

Precisely estimating a robot’s pose in a prior, global map is a fundamental capability for mobile robotics, e.g., autonomous driving or exploration in disaster zones. This task, however, remains challenging in unstructured, dynamic environments, where local features are not discriminative enough and global scene descriptors only provide coarse information. We therefore present SegMap: a map representation solution for localization and mapping based on the extraction of segments in 3D point clouds. Working at the level of segments offers increased invariance to view-point and local structural changes, and facilitates real-time processing of large-scale 3D data. SegMap exploits a single compact data-driven descriptor for performing multiple tasks: global localization, 3D dense map reconstruction, and semantic information extraction. The performance of SegMap is evaluated in multiple urban driving and search and rescue experiments. We show that the learned SegMap descriptor has superior segment retrieval capabilities, compared with state-of-the-art handcrafted descriptors. As a consequence, we achieve a higher localization accuracy and a 6% increase in recall over state-of-the-art handcrafted descriptors. These segment-based localizations allow us to reduce the open-loop odometry drift by up to 50%. SegMap is open-source available along with easy to run demonstrations.

Real-Time Rendering Method of Depth-Image-Based Multiple Reference Views for Integral Imaging Display

A real-time computer-generated integral imaging (CGII) display rendering method based on depth-image-based multiple reference viewpoints (MDIBR) is presented, and good three-dimensional (3D) image quality with the high frame rate can be achieved. The traditional rasterization pipeline and 3D image warping are used to accelerate CGII, and the proposed method is suitable for dynamic and complex 3D content generation for the 3D display. Qualitative and quantitative experiments are carried out to evaluate the feasibility of the proposed method. Experimental results show that the MDIBR method can achieve real-time integral imaging display with $80\times 80$ viewpoints based on large-scale 3D data with a million points.

ISPEED: a Scalable and Distributed In-Memory Based Spatial Query System for Large and Structurally Complex 3D Data.

The recent technological advancement in digital pathology has enabled 3D tissue-based investigation of human diseases at extremely high resolutions. Discovering and verifying spatial patterns among massive 3D micro-anatomic biological objects such as blood vessels and cells derived from 3D pathology image volumes plays a pivotal role in understanding diseases. However, the exponential increase of available 3D data and the complex structures of biological objects make it extremely difficult to support spatial queries due to high I/O, communication and computational cost for 3D spatial queries. In this demonstration, we present our scalable in-memory based spatial query system iSPEED for large-scale 3D data with complex structures. Low latency is managed by storing in memory with progressive compression including successive levels of detail on object level. On the other hand, low computational cost is achieved by pre-generation of global spatial indexes in memory and additional on-demand generation of indexing at run-time. Furthermore, iSPEED applies structural indexing on complex structured objects in multiple query types to gain performance advantage. During query processing, the memory footprint of iSPEED is minimal due to its indexing structure and progressive decompression on-demand. We demonstrate iSPEED query capability with three representative queries: 3D spatial joins, nearest neighbor and spatial proximity estimation on multiple datasets using a web based RESTful interface. Users can furthermore explore the input data structure, manage and adjust query pipeline parameters on the interface.PVLDB Reference Format:Hoang Vo, Yanhui Liang, Jun Kong, and Fusheng Wang. iSPEED: a Scalable and Distributed In-Memory Based Spatial Query System for Large and Structurally Complex 3D Data.

A Backprojection Slice Theorem for Tomographic Reconstruction.

Fast image reconstruction techniques are becoming important with the increasing number of scientific cases in high resolution micro and nano tomography. The processing of the large scale 3D data demands new mathematical tools for the tomographic reconstruction. Due to the high computational complexity of most current algorithms, big data sizes demands powerful hardware and more sophisticated numerical techniques. Several reconstruction algorithms are dependent on a mathematical tool called backprojection (a transposition process). A conventional implementation of the backprojection operator has cubic computational complexity. In the present manuscript we propose a new fast backprojection operator for the processing of tomographic data, providing a low-cost algorithm for this task. We compare our formula against other fast transposition techniques, using real and simulated large data sets.

ISPEED: an Efficient In-Memory Based Spatial Query System for Large-Scale 3D Data with Complex Structures.

Recent advances in digital pathology make it possible to support 3D tissue-based investigation of human diseases at extremely high resolutions. Exploring spatial relationships and patterns among massive 3D micro-anatomic biological objects such as blood vessels and cells derived from 3D pathology image volumes plays a critical role in studying human diseases. In this paper, we present our work on building an effective and scalable in-memory based spatial query system iSPEED for large-scale 3D data with complex structures. To achieve low latency, iSPEED stores data in memory with effective progressive compression for each 3D object with successive levels of detail. To minimize search space and computation cost, iSPEED pregenerates global spatial indexes in memory and employs on-demand indexing at run-time. In particular, iSPEED exploits structural indexing for complex structured objects in distance based queries. iSPEED provides a 3D spatial query engine that can be invoked on-demand to run many instances in parallel implemented with, but not limited to, MapReduce. iSPEED builds in-memory indexes and decompresses data on-demand, which has minimal memory footprint. We evaluate iSPEED with two representative queries: 3D spatial joins and 3D spatial proximity estimation. Our experiments demonstrate that iSPEED significantly improves the performance over traditional non-memory based spatial query systems.

Research on Image Storage Based on 3D Point Cloud

3D technology has been widely used in the fields of architecture, medical image, cultural relic protection, filmproduction and 3D game. The polygon mesh is the most commonly used method for expressing the 3D model, itconnects the points on the surface of the 3D model into polygons as a unit, which can express complex surfaces andprovide strong adaptability. Especially in the use of triangular mesh the most widely used. But the triangular griddisplay, simplifi ed, progressive transmission algorithm is not suitable for large-scale data sets. The triangular mesh datafi le described in this paper with the boundary ball data fi le conversion system will represent and gradually compressthese triangular meshes. This representation is not only compact, but also can be quickly calculated, its biggest featureis the high compression ratio, saving a lot of time and space, which makes it suitable for large-scale data sets. Itsexecution program can serve large-scale 3D data processing projects. This article has demonstrated that this system canbe converted to include a large order of magnitude polygon model.

Sparse 3D directional vertices vs continuous 3D curves: Efficient 3D surface matching and its application for single model face recognition

Traditionally, point clouds and meshes are used to represent and match 3D shapes, which often cannot meet the computational speed and storage space requirements in many 3D data matching and retrieval applications. In this paper, we present a novel 3D directional vertices (3D2V) approach to efficiently represent and match 3D surfaces by much fewer sparsely distributed structured vertices that carry structural information transferred from their deleted neighbouring points. A 3D2V conversion and similarity measurement method is developed to compute the distance between two different 3D2Vs. The performance of the proposed method is evaluated on 3D face recognition using Face Recognition Grand Challenge v2.0 (FRGC v2.0) and GavabDB databases and compared with the curve-based benchmark method. The experimental results demonstrate that the proposed 3D2V method can significantly reduce the data storage requirement and computation time with a moderate increase of accuracy at the same time. It provides a new tool for developing fast 3D surface matching algorithms for large scale 3D data classification and retrieval.

Multiple ray cluster rendering for interactive integral imaging system

In this paper, we present an efficient Computer Generated Integral Imaging (CGII) method, called multiple ray cluster rendering (MRCR). Based on the MRCR, an interactive integral imaging system is realized, which provides accurate 3D image satisfying the changeable observers' positions in real time. The MRCR method can generate all the elemental image pixels within only one rendering pass by ray reorganization of multiple ray clusters and 3D content duplication. It is compatible with various graphic contents including mesh, point cloud, and medical data. Moreover, multi-sampling method is embedded in MRCR method for acquiring anti-aliased 3D image result. To our best knowledge, the MRCR method outperforms the existing CGII methods in both the speed performance and the display quality. Experimental results show that the proposed CGII method can achieve real-time computational speed for large-scale 3D data with about 50,000 points.