Real 3D Datasets Research Articles

Complementary pseudo multimodal feature for point cloud anomaly detection

Point cloud anomaly detection is steadily emerging as a promising research area. Recognizing the importance of feature descriptiveness in this task, this study introduces the Complementary Pseudo Multimodal Feature (CPMF), which combines local geometrical information extracted by 3D handcrafted descriptors with global semantic information extracted from 2D pre-trained neural networks. Specifically, to leverage 2D pre-trained neural networks for point-wise feature extraction, this study projects original point clouds into multi-view images. These images are then fed into a pre-trained 2D neural network for informative 2D modality feature extraction. Following the 2D–3D correspondence, the multi-view 2D modality features are projected back to 3D space and aggregated to obtain point-wise 2D modality features. Finally, the point-wise 3D and 2D modality features are fused to derive the CPMF for point cloud anomaly detection. Extensive experiments conducted on MVTec 3D and Real3D datasets demonstrate the complementary capacity between 2D and 3D modality features and the effectiveness of CPMF. Notably, CPMF achieves a significantly higher object-level AUROC of 95.15% compared to other methods on the MVTec 3D benchmark. Code is available at https://github.com/caoyunkang/CPMF.

Reverse time migration surface offset gathers by attribute migration and constrained least-squares inversion

Abstract Surface offset gathers (SOGs) are crucial for velocity updating in seismic data migration and muting the stretched waveform in shallow parts of migrated seismic data. Kirchhoff-based migration methods can output SOGs because they take every seismic trace as the input. As a result, the imaging results can be rearranged according to the offset value of the input data to obtain SOGs. However, regarding more accurate wave equation methods (such as reverse time migration, RTM), the SOGs cannot be obtained directly with a single migration calculation. Attribute migration is an easy-to-implement method to output SOGs of wave equation migration methods. It calculates the offset value of each imaging point by dividing the migration result modulated by the offset attribute with the conventional migration result. However, the division error may cause the calculation result outside the given offset value. This paper proposes a constrained least-squares inversion method for stable calculation of the RTM offset range. The method ensures that the calculated results are within the given offset range. We tested the method against the direct division and least-squares methods without constraints using the Marmousi model and a real 3D dataset. We show that the SOGs obtained by the constrained least-squares inversion attribute migration method had the same characteristics as those derived using the division-based attribute migration method, but they also had higher vertical resolution, better energy distribution, and improved lateral continuity. In a further study, we expect to develop a stable and direct method to efficiently calculate SOGs for RTM and an iterative closed loop for RTM velocity updating.

Zero-Shot Learning on 3D Point Cloud Objects and Beyond

Zero-shot learning, the task of learning to recognize new classes not seen during training, has received considerable attention in the case of 2D image classification. However, despite the increasing ubiquity of 3D sensors, the corresponding 3D point cloud classification problem has not been meaningfully explored and introduces new challenges. In this paper, we identify some of the challenges and apply 2D Zero-Shot Learning (ZSL) methods in the 3D domain to analyze the performance of existing models. Then, we propose a novel approach to address the issues specific to 3D ZSL. We first present an inductive ZSL process and then extend it to the transductive ZSL and Generalized ZSL (GZSL) settings for 3D point cloud classification. To this end, a novel loss function is developed that simultaneously aligns seen semantics with point cloud features and takes advantage of unlabeled test data to address some known issues (e.g., the problems of domain adaptation, hubness, and data bias). While designed for the particularities of 3D point cloud classification, the method is shown to also be applicable to the more common use-case of 2D image classification. An extensive set of experiments is carried out, establishing state-of-the-art for ZSL and GZSL on synthetic (ModelNet40, ModelNet10, McGill) and real (ScanObjectNN) 3D point cloud datasets.

Adapted human pose: monocular 3D human pose estimation with zero real 3D pose data

The ultimate goal for an inference model is to be robust and functional in real life applications. However, training vs. test data domain gaps often negatively affect model performance. This issue is especially critical for the monocular 3D human pose estimation problem, in which 3D human data is often collected in a controlled lab setting. In this paper, we focus on alleviating the negative effect of domain shift in both appearance and pose space for 3D human pose estimation by presenting our adapted human pose (AHuP) approach. AHuP is built upon two key components: (1) semantically aware adaptation (SAA) for the cross-domain feature space adaptation, and (2) skeletal pose adaptation (SPA) for the pose space adaptation which takes only limited information from the target domain. By using zero real 3D human pose data, one of our adapted synthetic models shows comparable performance with the SOTA pose estimation models trained with large scale real 3D human datasets. The proposed SPA can be also employed independently as a light-weighted head to improve existing SOTA models in a novel context. A new 3D scan-based synthetic human dataset called ScanAva+ is also going to be publicly released with this work.

Virtual simulation benchmark for the evaluation of simultaneous localization and mapping and 3D reconstruction algorithm uncertainty

Simultaneous localization and mapping (SLAM) algorithms allow us to obtain a unique 3D shape and 3D sensor trajectory by combining partial scans obtained by moving a 3D scanner. The performances of these algorithms are significantly affected by experimental conditions, characteristics of the target and values of the parameters of the reconstruction algorithm. Therefore, the uncertainty and reliability of SLAM techniques need to be assessed before their application, e.g. for robot navigation, autonomous vehicles or industrial fields. To evaluate the uncertainty of these algorithms, specific datasets containing 3D scans, with the possibility to control different conditions, e.g. sensor trajectory, depth or color noise, sensor velocity and framerate, are necessary. In this article, we present a procedure to obtain virtual datasets with complete control of the environment, 3D sensor and trajectory conditions, starting from any real 3D dataset acquisition, characterized by a sufficiently low uncertainty. These datasets can be generated to test the effect of SLAM algorithm parameters to determine the best parameters to be used to exploit the algorithm characteristics to obtain the best result in each operating context. The advantage of this procedure is the possibility to perfectly control each condition and to evaluate its effect on the final result. This procedure was applied to two reconstruction algorithms as examples; namely, the Open3D reconstruction tool and ElasticFusion. The results demonstrate that the setting of algorithm parameters, e.g. the tolerance on depth correspondence between frames or the number of fragments, or the change in number of frames acquired, can have a strong influence on the resulting 3D reconstruction and trajectory. Moreover, the effect of not closing the loop trajectory on reconstruction performance is quantified for different application scenarios.

Map Comparison Methods for Three‐Dimensional Space and Time Voxel Data

Map comparisons in three‐dimensional space (3D) and 3D time series (4D) are becoming a necessity with increased availability of multidimensional data and model simulation outputs. Therefore, this research study extends the two‐dimensional (2D) map comparison methods with the aim to propose a suite of 3D approaches such as 3D Kappa, 3D Fuzzy, and 4D Fuzzy Kappa coefficients specifically designed to perform with voxel data. These proposed approaches can account for fuzziness where small categorical differences in 3D space or space‐time are given a degree of similarity instead of a binary similarity value. The developed approaches are tested using different voxel data sets: (a) hypothetical with two and four classes to confirm the methods produce expected results, (b) voxelized LiDAR data to demonstrate the comparison of real 3D data sets, (c) soil horizon voxel data sets to conduct a sensitivity analysis of 3D voxel window sizes, (d) 4D outputs from an agent‐based forest‐fire smoke model to demonstrate the 4D Fuzzy Kappa coefficient and a sensitivity analysis of 4D voxel window size. The obtained results indicate that 3D and 4D voxel data comparisons are feasible allowing for further work on comparison of 3D data and evaluation of multidimensional models.

Point Cloud Stacking: A Workflow to Enhance 3D Monitoring Capabilities Using Time-Lapse Cameras

The emerging use of photogrammetric point clouds in three-dimensional (3D) monitoring processes has revealed some constraints with respect to the use of LiDAR point clouds. Oftentimes, point clouds (PC) obtained by time-lapse photogrammetry have lower density and precision, especially when Ground Control Points (GCPs) are not available or the camera system cannot be properly calibrated. This paper presents a new workflow called Point Cloud Stacking (PCStacking) that overcomes these restrictions by making the most of the iterative solutions in both camera position estimation and internal calibration parameters that are obtained during bundle adjustment. The basic principle of the stacking algorithm is straightforward: it computes the median of the Z coordinates of each point for multiple photogrammetric models to give a resulting PC with a greater precision than any of the individual PC. The different models are reconstructed from images taken simultaneously from, at least, five points of view, reducing the systematic errors associated with the photogrammetric reconstruction workflow. The algorithm was tested using both a synthetic point cloud and a real 3D dataset from a rock cliff. The synthetic data were created using mathematical functions that attempt to emulate the photogrammetric models. Real data were obtained by very low-cost photogrammetric systems specially developed for this experiment. Resulting point clouds were improved when applying the algorithm in synthetic and real experiments, e.g., 25th and 75th error percentiles were reduced from 3.2 cm to 1.4 cm in synthetic tests and from 1.5 cm to 0.5 cm in real conditions.

Efficient processing of all-k-nearest-neighbor queries in the MapReduce programming framework

Numerous modern applications, from social networking to astronomy, need efficient answering of queries on spatial data. One such query is the All k Nearest-Neighbor Query, or k Nearest-Neighbor Join, that takes as input two datasets and, for each object of the first one, returns the k nearest-neighbors from the second one. It is a combination of the k nearest-neighbor and join queries and is computationally demanding. Especially, when the datasets involved fall in the category of Big Data, a single machine cannot efficiently process it. Only in the last few years, papers proposing solutions for distributed computing environments have appeared in the literature. In this paper, we focus on parallel and distributed algorithms using the Apache Hadoop framework. More specifically, we focus on an algorithm that was recently presented in the literature and propose improvements to tackle three major challenges that distributed processing faces: improvement of load balancing (we implement an adaptive partitioning scheme based on Quadtrees), acceleration of local processing (we prune points during calculations by utilizing plane-sweep processing), and reduction of network traffic (we restructure and reduce the output size of the most demanding phase of computation). Moreover, by using real 2D and 3D datasets, we experimentally study the effect of each improvement and their combinations on performance of this literature algorithm. Experiments show that by carefully addressing the three aforementioned issues, one can achieve significantly better performance. Thereby, we conclude to a new scalable algorithm that adapts to the data distribution and significantly outperforms its predecessor. Moreover, we present an experimental comparison of our algorithm against other well-known MapReduce algorithms for the same query and show that these algorithms are also significantly outperformed.

Geostatistical interpolation of non-stationary seismic data

The problem of sparsely collected seismic data is one of the main issues in reflection seismology, because most advanced data processing techniques require a dense and regular seismic data grid. We present a geostatistical seismic data interpolation technique based on sequential stochastic simulations with local structural anisotropies. This technique, contrary to conventional existing data-driven seismic interpolation approaches based on sparsity, prediction filters, or rank-reduction, predicts the value of seismic amplitudes at non-sampled locations by exploiting the statistics of the recorded amplitudes, which are used as experimental data for the geostatistical interpolation in the original data domain. Local mean and variance are computed on-the-fly to define intervals of the global conditional distribution function, from where amplitude values are stochastically simulated. The parameters to define subsets of experimental data from which mean and variance are calculated are given by local variogram models, which in turn are obtained from a local dip and azimuth estimation in the t-x-y domain. The geostatistical seismic data interpolation technique is applied to synthetic and real 2D and 3D datasets in both post- and pre-stack domains. Besides being computationally cheaper than other methods, because the interpolation is carried out directly in the original data domain, the proposed technique provides a local quantitative analysis of the reliability of the interpolated seismic samples, which can be exploited in following processing steps.

Broadband Least-Squares Wave-Equation Migration

We introduce an effective iterative Least-Squares Wave-Equation Migration (LS-WEM) solution for broadband imaging. Least-Squares Migration (LSM) solutions are designed to produce images of the subsurface corrected for wavefield distortions caused by acquisition and propagation effects. They implicitly solve for the earth reflectivity by means of data residual reduction in an iterative fashion, which usually demands intensive computation. The LS-WEM is implemented using a visco-acoustic anisotropic one-way wave-equation wavefield propagator that is able to fully utilize both the broader seismic bandwidth and the high-resolution velocity information from Full Waveform Inversion (FWI). Our implementation combines the one-way extrapolator with fast linear inversion solvers into an efficient migration inversion system. Application to the 2D Sigsbee2b synthetic model improves the sub-salt illumination by balancing the image amplitudes and reducing the effects of the shadow zones, enhances temporal resolution by broadening the frequency spectrum, balances the wavenumber content and improves images of faults and dipping salt flanks. In addition, LS-WEM converges rapidly to the true solution, reducing the data residuals by 90% in only four iterations. Application to real 3D datasets from the Gulf of Mexico and the North Sea demonstrates high-resolution imaging with reduced acquisition footprint effects, improved spatial frequency content, and better structural imaging at all depths.

An overview of reproducible 3D seismic data processing and imaging using Madagascar

We present an overview of reproducible 3D seismic data processing and imaging using the Madagascar open-source software package. So far, there has been a limited number of studies on the processing of real 3D data sets using open-source software packages. Madagascar with its wide range of individual programs and tools available provides the capability to fully process 3D seismic data sets. The goal is to provide a streamlined illustration of the approach for the implementation of 3D seismic data processing and imaging using the Madagascar open-source software package. A brief introduction is first given to the Madagascar open-source software package and the publicly available 3D Teapot Dome seismic data set. Several processing steps are applied to the data set, including amplitude gaining, ground roll attenuation, muting, deconvolution, static corrections, spike-like random noise elimination, normal moveout (NMO) velocity analysis, NMO correction, stacking, and band-pass filtering. A 3D velocity model in depth is created using Dix conversion and time-to-depth scaling. Three-dimensional poststack depth migration is then performed followed by [Formula: see text]-[Formula: see text] deconvolution and structure-enhancing filtering of the migrated image to suppress random noise and enhance the useful signal. We show that Madagascar, as a powerful open-source environment, can be used to construct a basic workflow to process and image 3D seismic data in a reproducible manner.

Novel fuzzy clustering scheme for 3D wireless sensor networks

In this paper, we consider the sensor-energy optimization in 3D Wireless Sensor Networks (WSNs), which determines an optimal topology of sensors to prolong the network lifetime and reduce the energy expenditure. A new mathematical model for clustering in 3D WSN, considering energy consumption, constraints of communication and a 3D energy function, is presented. Using the Lagrange multiplier method, solutions of the model consisting of cluster centres and the membership matrix are computed and used in the new algorithm, called FCM-3 WSN. Experimental validation on real 3D datasets demonstrates that FCM-3 WSN outperforms the relevant methods, namely, Low-Energy Adaptive Clustering Hierarchy (LEACH), Centralized LEACH (LEACH-C), Single-hop Clustering and Energy-Efficient Protocol (SCEEP), Hybrid-Low Energy Adaptive Clustering Hierarchy (H-LEACH), K-Means and Fuzzy C-Means (FCM).

Improving lifetime and network connections of 3D wireless sensor networks based on fuzzy clustering and particle swarm optimization

3D wireless sensor network (3D-WSN) has attracted significant interests in recent years due to its applications in various disciplinary fields such as target detection, object tracking, and security surveillance. An important problem in 3D WSN is the sensor energy optimization which determines a topology of sensors to prolong the network lifetime and energy expenditure. The existing methods for dealing with this matter namely low energy adaptive clustering hierarchy, LEACH-centralized, K-Means, single hop clustering and energy efficient protocol, hybrid-LEACH and fuzzy C-means organize the networks into clusters where non-cluster head nodes mainly carry out sensing tasks and send the information to the cluster head, while cluster head collect data from other nodes and send to the base station (BS). Although these algorithms reduce the total energy consumption of the network, they also create a large number of network disconnect which refers to the number of sensors that cannot connect to its cluster head and the number of cluster heads that cannot connect to the BS. In this paper, we propose a method based on fuzzy clustering and particle swarm optimization to handle this problem. Experimental validation on real 3D datasets indicates that the proposed method is better than the existing methods.

Direct 4D PET MLEM reconstruction of parametric images using the simplified reference tissue model with the basis function method for [11C]raclopride

This work assesses the one-step late maximum likelihood expectation maximization (OSL-MLEM) 4D PET reconstruction algorithm for direct estimation of parametric images from raw PET data when using the simplified reference tissue model with the basis function method (SRTM-BFM) for the kinetic analysis. To date, the OSL-MLEM method has been evaluated using kinetic models based on two-tissue compartments with an irreversible component. We extend the evaluation of this method for two-tissue compartments with a reversible component, using SRTM-BFM on simulated 3D + time data sets (with use of [11C]raclopride time-activity curves from real data) and on real data sets acquired with the high resolution research tomograph. The performance of the proposed method is evaluated by comparing voxel-level binding potential (BPND) estimates with those obtained from conventional post-reconstruction kinetic parameter estimation. For the commonly chosen number of iterations used in practice, our results show that for the 3D + time simulation, the direct method delivers results with lower %RMSE at the normal count level (decreases of 9–10 percentage points, corresponding to a 38–44% reduction), and also at low count levels (decreases of 17–21 percentage points, corresponding to a 26–36% reduction). As for the real 3D data set, the results obtained follow a similar trend, with the direct reconstruction method offering a 21% decrease in %CV compared to the post reconstruction method at low count levels. Thus, based on the results presented herein, using the SRTM-BFM kinetic model in conjunction with the OSL-MLEM direct 4D PET MLEM reconstruction method offers an improvement in performance when compared to conventional post reconstruction methods.

Direct diffeomorphic reparameterization for correspondence optimization in statistical shape modeling

In this paper, we propose an efficient optimization approach for obtaining shape correspondence across a group of objects for statistical shape modeling. With each shape represented in a B-spline based parametric form, the correspondence across the shape population is cast as an issue of seeking a reparameterization for each shape so that a quality measure of the resulting shape correspondence across the group is optimized. The quality measure is the description length of the covariance matrix of the shape population, with landmarks sampled on each shape. The movement of landmarks on each B-spline shape is controlled by the reparameterization of the B-spline shape. The reparameterization itself is also represented with B-splines and B-spline coefficients are used as optimization parameters. We have developed formulations for ensuring the bijectivity of the reparameterization. A gradient-based optimization approach is developed, including techniques such as constraint aggregation and adjoint sensitivity for efficient, direct diffeomorphic reparameterization of landmarks to improve the group-wise shape correspondence. Numerical experiments on both synthetic and real 2D and 3D data sets demonstrate the efficiency and effectiveness of the proposed approach.

Scalable visibility color map construction in spatial databases

Recent advances in 3D modeling provide us with real 3D datasets to answer queries, such as “What is the best position for a new billboard?” and “Which hotel room has the best view?” in the presence of obstacles. These applications require measuring and differentiating the visibility of an object (target) from different viewpoints in a dataspace, e.g., a billboard may be seen from many points but is readable only from a few points closer to it. In this paper, we formulate the above problem of quantifying the visibility of (from) a target object from (of) the surrounding area with a visibility color map (VCM). A VCM is essentially defined as a surface color map of the space, where each viewpoint of the space is assigned a color value that denotes the visibility measure of the target from that viewpoint. Measuring the visibility of a target even from a single viewpoint is an expensive operation, as we need to consider factors such as distance, angle, and obstacles between the viewpoint and the target. Hence, a straightforward approach to construct the VCM that requires visibility computation for every viewpoint of the surrounding space of the target is prohibitively expensive in terms of both I/Os and computation, especially for a real dataset comprising thousands of obstacles. We propose an efficient approach to compute the VCM based on a key property of the human vision that eliminates the necessity for computing the visibility for a large number of viewpoints of the space. To further reduce the computational overhead, we propose two approximations; namely, minimum bounding rectangle and tangential approaches with guaranteed error bounds. Our extensive experiments demonstrate the effectiveness and efficiency of our solutions to construct the VCM for real 2D and 3D datasets.

Morpho-volumetric assessment of the right ventricle: a comparison between three-dimensional echocardiography and cardiac magnetic resonance

The morpho-volumetric assessment of the right ventricle (RV) is useful in the diagnosis and prognosis of many congenital and acquired cardiovascular diseases. The complexity of the right ventricular shape does not allow an adequate and satisfactory evaluation of the RV. Two-dimensional (2D) echocardiography is the most used tool for the assessment of RV function, as it is a noninvasive, reproducible and widespread technique, but the geometric assumptions required for measurement of RV volumes reduce its accuracy. At present, cardiac magnetic resonance is the gold standard for the evaluation of RV volumes because it does not need any assumption and is reproducible, though not always available. This imaging tool allows to follow-up patients and to monitor drug therapy. On the other hand, three-dimensional (3D) echocardiography overcomes several limitations of 2D echocardiography, in that it provides an estimation of RV volumes obtained through a real 3D data set, obtaining sections which cannot be technically achieved with 2D echocardiography. It is therefore useful to compare data on RV volumes and function obtained with 3D echocardiography versus cardiac magnetic resonance, mainly considering the lower costs and higher portability and availability of echocardiography.

Optimum split-step Fourier 3D depth migration: Developments and practical aspects

We present an efficient scheme for depth extrapolation of wide-angle 3D wavefields in laterally heterogeneous media. The scheme improves the so-called optimum split-step Fourier method by introducing a frequency-independent cascaded operator with spatially varying coefficients. The developments improve the approximation of the optimum split-step Fourier cascaded operator to the exact phase-shift operator of a varying velocity in the presence of strong lateral velocity variations, and they naturally lead to frequency-dependent varying-step depth extrapolations that reduce computational cost significantly. The resulting scheme can be implemented alternatively in spatial and wavenumber domains using fast Fourier transforms (FFTs). The accuracy of the first-order approximate algorithm is similar to that of the second-order optimum split-step Fourier method in modeling wide-angle propagation through strong, laterally varying media. Similar to the optimum split-step Fourier method, the scheme is superior to methods such as the generalized screen and Fourier finite difference. We demonstrate the scheme’s accuracy by comparing it with 3D two-way finite-difference modeling. Comparisons with the 3D prestack Kirchhoff depth migration of a real 3D data set demonstrate the practical application of the proposed method.

An integrated procedure for migration velocity analysis in complex structures of thrust belts

Velocity analysis is a key part in seismic depth imaging because it has a direct effect on the quality of the depth-migrated seismic images. In complex structures, lateral velocity variations and complicated geometry of structures make the velocity analysis a challenging task. A variety of techniques using pre-stack seismic data exists for migration velocity analysis, including tomographic inversion methods which update velocity globally and are more efficient for complex situations. I present a procedure for migration velocity analysis in complex fault-folded structures using two types of depth tomography: cell-based tomography and layer-based tomography. The proposed method has 4 main stages including initial velocity model generation, large wavelength (regional) velocity update, short wavelength velocity update and final model generation. Pre-stack depth migration has been used to measure the quality of the estimated velocity at each stage. Relevant a priori information has been proposed for each stage to constrain the velocity analysis. The velocity analysis starts from regional updates and general structural framework reconstruction, then deals with complex structures in detail and finally finishes with a high-resolution velocity field that is consistent with the complex geometry as well as the rock types. Knowledge of structural style, well-derived information including logs, vertical seismic profiling data and the geologic markers together with detailed seismic interpretation are the main sources of information to control the inversion process of migration velocity analysis and ensure convergence to the correct answer. At the second and third stages of the proposed procedure, cell-based tomography is applied and at the last stage horizon-based tomography has been applied. The method has been tested on a real 3D dataset from the Iranian Zagros fold-thrust belt with strong topographic variations. The area has structural complexity including different deformation styles at different depth levels, fault-related folds, and various thicknesses of incompetent rock types such as salt and thick layers of competent rocks such as carbonates. The results of velocity analysis are completely consistent with the structural style of the area. It includes more detail about the rock types. The derived velocity model shows deformed incompetent layers in the shallow section and dependency of carbonate velocity on depositional environment at deeper levels. The final velocity model is structurally consistent and provides better focusing of the seismic images.