Visualization Of Point Clouds Research Articles

The Performance of the Optical Flow Field based Dense Image Matching for UAV Imagery

Abstract. With the rapid development of sensing platforms, unmanned aerial vehicle (UAV)-based mapping has become increasingly popular because of its economic efficiency and flexibility, especially for providing 3D information to support urban growth monitoring and change detection to meet sustainable development goals (SDGs). This paper presents an improved optical flow field-based dense matching algorithm (OFFDM) for low-altitude UAV images based on the Ph.D. thesis of Yuan (Yuan, 2018). First, high-precision seed points were used to compute the optical flow field within stereo pairs, effectively minimizing redundant calculations during the fine-matching phase. Second, a fine-matching approach, integrating multiple constraints, was applied to refine the coarse matching results based on the optical flow field. Extensive dense matching experiments on UAV low-altitude aerial imagery assessed the performance of OFFDIM across four dimensions: 3D point cloud visualization, matching success rate, precision, and reliability. Extensive experiments on low-altitude UAV imagery, characterized by a resolution of 7cm per pixel over a 10,608×8,608 pixel dimension and a 60% forward overlap, evaluate the OFFDM's efficacy. The quantitative evaluation revealed that the proposed method achieved an accuracy of ±0.7 pixels in image coordinates and ±20 cm on the ground, with a matching success rate exceeding 97%. The processing time was approximately 272 seconds for handling one single stereo pair. When compared to the widely adopted PMVS algorithm, known for its effectiveness in dense matching for UAV images, the proposed method demonstrated higher completeness and improved matching efficiency by more than five times. These results demonstrated that the proposed approach is more suitable for dense matching on UAV imagery-based high-precision 3D spatial data extraction, supporting global mapping tasks more effectively.

Maintenance Approaches Using 3D Scanning Point Cloud Visualization, and BIM+ Data Management: A Case Study of Dahei Mountain Buildings

This study investigates the utilization of 3D laser scanning, GPS, BIM and GIS technologies for safeguarding historical wooden structures. As a case study, it focuses on two historic Chinese wooden buildings, the Guanyin Pavilion and Tangwang Palace, renowned for their exceptional historical significance and intricate timber architecture. The Guanyin Pavilion was subjected to high-precision modeling and tilt monitoring, which found that certain columns had a maximum tilt angle of 0.7892°. Additionally, a general tilt tendency was seen across various structural elements. The measurements from the point cloud data exhibited an error range of ±10 mm. However, the relative errors in smaller components approached 2.5%, which necessitated regular verification through human measurements. Using GIS and BIM technology in Tangwang Palace facilitated the meticulous documentation and dynamic supervision of architectural components, encompassing historical development and present state. The maintenance records revealed persistent problems with the roof tiles and wooden beams, exposing structural vulnerabilities. During various maintenance intervals, it was observed that the roof tiles were frequently loose and separated, indicating potential flaws in the design or materials used. The application of these modern technologies enhances the effectiveness and precision of conservation efforts and provides a robust scientific foundation for the continuous monitoring and study of cultural heritage.

Random Data Distribution for Efficient Parallel Point Cloud Processing

Abstract. Current point cloud data management systems and formats are heavily specialized and targeted solely towards visualization purposes and fail to address the diverse needs of progressive point cloud workflows like for example semantic segmentation using machine learning. We therefore propose a distributed data infrastructure for dynamic point cloud data management that can support interactive real-time visualization at scale while simultaneously serving as a platform for analytical tasks. By introducing random data distribution, we show that simple query fragmentation and efficient and effective parallelism at scale are possible. At the same time, arbitrary queries in space and time can be efficiently run over the infrastructure including query semantics which returns only a random sample of the query results or preferred points based on an importance dimension calculated, for example, from a local point density information as commonly done in point cloud visualization. To cope with the unknown amount of user-specific attributes and to support even multiple ways of deciding the importance of a given point (ground point removal, coverage of space, random subset) the system is designed to support all of them transparently as multidimensional range queries backed by spatial indices.

A Survey on Non-photorealistic Rendering Approaches for Point Cloud Visualization.

Point clouds are widely used as a versatile representation of 3D entities and scenes for all scale domains and in a variety of application areas, serving as a fundamental data category to directly convey spatial features. However, due to point sparsity, lack of structure, irregular distribution, and acquisition-related inaccuracies, results of point cloud visualization are often subject to visual complexity and ambiguity. In this regard, non-photorealistic rendering can improve visual communication by reducing the cognitive effort required to understand an image or scene and by directing attention to important features. In the last 20 years, this has been demonstrated by various non-photorealistic rendering approaches that were proposed to target point clouds specifically. However, they do not use a common language or structure for assessment which complicates comparison and selection. Further, recent developments regarding point cloud characteristics and processing, such as massive data size or web-based rendering are rarely considered. To address these issues, we present a survey on non-photorealistic rendering approaches for point cloud visualization, providing an overview of the current state of research. We derive a structure for the assessment of approaches, proposing seven primary dimensions for the categorization regarding intended goals, data requirements, used techniques, and mode of operation. We then systematically assess corresponding approaches and utilize this classification to identify trends and research gaps, motivating future research in the development of effective non-photorealistic point cloud rendering methods.

Using t-distributed stochastic neighbor embedding for visualization and segmentation of 3D point clouds of plants

Using t-distributed stochastic neighbor embedding for visualization and segmentation of 3D point clouds of plants

Multi feature-rich synthetic colour to improve human visual perception of point clouds

Although point features have shown their usefulness in classification with Machine Learning, point cloud visualization enhancement methods focus mainly on lighting. The visualization of point features helps to improve the perception of the 3D environment. This paper proposes Multi Feature-Rich Synthetic Colour (MFRSC) as an alternative non-photorealistic colour approach of natural-coloured point clouds. The method is based on the selection of nine features (reflectance, return number, inclination, depth, height, point density, linearity, planarity, and scattering) associated with five human perception descriptors (edges, texture, shape, size, depth, orientation). The features are reduced to fit the RGB display channels. All feature permutations are analysed according to colour distance with the natural-coloured point cloud and Image Quality Assessment. As a result, the selected feature permutations allow a clear visualization of the scene's rendering objects, highlighting edges, planes, and volumetric objects. MFRSC effectively replaces natural colour, even with less distorted visualization according to BRISQUE, NIQUE and PIQE. In addition, the assignment of features in RGB channels enables the use of MFRSC in software that does not support colorization based on point attributes (most commercially available software). MFRSC can be combined with other non-photorealistic techniques such as Eye-Dome Lighting or Ambient Occlusion.

Digital characterization of the surface texture of chinese classical garden rockery based on point cloud visualization: small-rock mountain retreat

The Rockery is often a key element of a Classical Chinese Garden. It’s exquisite detailed physical characteristics a major contributor to artistic value, aesthetic appeal, and the carrier of historical and cultural heritage values. Poets and scholars have often described the beauty of these places in classical gardens in qualitative terms but lacked the quantitative tools to provide replicable metric descriptions. The highly complex forms and surfaces, irregularity, and fragility of garden rockeries has challenged authors to accurately describe the characteristics of these qualities using traditional methods and tools. This article presents a new method of digital characterization approach based on laser scanning and point cloud visualization, which can quantitatively detect and represent the pattern of rockery surface textures. It offers a replicable accurate quantitative descriptor of the Classical Chinese rockery. The Small-Rock Mountain Retreat, a nationally protected rockery garden in China, has been used as a case study. It contains original historic elements and more recently restored areas. Two characteristics of rockery surfaces, including the well-proportioned density and space, and the proper contrast between solid and void, were analyzed by examining four attributes: (1) surface complexity; (2) contour curvature; (3) shape variation; and (4) the interweaving of lightness and darkness. The findings demonstrate that, despite some similarities between the restored portion of the rockery and the historical remnants, there are variances in the richness of the details and the balanced distribution of shape change. The digital characterization approach introduced in this article offers a new perspective for recording and in turn safeguarding Chinese garden rockeries and other irregular cultural heritage objects.

Interactive Visualization of Large Point Clouds Using an Autotuning Multiresolution Out-Of-Core Strategy

Abstract Due to the increasingly large amount of data acquired into point clouds, from LiDAR (Light Detection and Ranging) sensors and 2D/3D sensors, massive point clouds processing has become a topic with high interest for several fields. Current client-server applications usually use multiresolution out-of-core proposals; nevertheless, the construction of the data structures required is very time-consuming. Furthermore, these multiresolution approaches present problems regarding point density changes between different levels of detail and artifacts due to the rendering of elements entering and leaving the field of view. We present an autotuning multiresolution out-of-core strategy to avoid these problems. Other objectives are reducing loading times while maintaining low memory requirements, high visualization quality and achieving interactive visualization of massive point clouds. This strategy identifies certain parameters, called performance parameters, and defines a set of premises to obtain the goals mentioned above. The optimal parameter values depend on the number of points per cell in the multiresolution structure. We test our proposal in our web-based visualization software designed to work with the structures and storage format used and display massive point clouds achieving interactive visualization of point clouds with more than 27 billion points.

Progress of Photonic-Crystal Surface-Emitting Lasers: A Paradigm Shift in LiDAR Application

Nowadays, the flurry of autonomous vehicles is in full swing regarding light detection and ranging (LiDAR) and depth perception. For such visual perception, light plays an important role. We human beings recognize and distinguish surrounding details when the eye focuses light on the retina. For the LiDAR system, pulsed lasers are employed to measure the relevant range. Thus, appropriate light sources with high performance are in urgent demand. Auspiciously, a revolutionary semiconductor laser technology, namely the photonic-crystal surface-emitting laser (PCSEL), emerges over the past two decades. PCSEL exhibits not only a symmetric beam profile with narrow beam divergence but also a high-power operation with controllability. Therefore, it may be the holy grail for an ultracompact time-of-flight (ToF) LiDAR system. Hereupon, comprehensive analyses of PCSEL-relevant scientific publications and patent documents are conducted. We thereby review the development progress of PCSEL technology. Moreover, a systematic simulation is performed, providing real-time visualization of relevant point clouds with different beam divergence. PCSEL technology with unprecedented merits indeed turns a new leaf and a paradigm shift in LiDAR application is ongoing. It is believed that a lens-free and adjustment-free ultracompact apparatus in simplicity can be expected.

MIXED REALITY VISUALIZATION OF POINT CLOUDS FOR SUPPORTING TERRESTRIAL LASER SCANNING

Abstract. 3D point clouds from terrestrial laser scanners (TLS) are used in a variety of fields and applications. To acquire high-quality point clouds that have enough point density, small scanning errors, and no lack of points in important regions, appropriate scan planning, including determination of scanner positions and scan conditions, is required. Currently, planning is supported by knowledge and experience of skilled workers, and it is difficult to ensure the quality of acquired point clouds. In this study, we propose a system for visualization of point clouds to support the acquisition of high-quality point clouds using TLS. The system allows the user to see and check the quality of scanned TLS point clouds and unscanned regions intuitively by superimposing the point clouds onto the real world using a mixed reality (MR) device. In addition, the system supports finding the next best scanner position for additional laser scans based on predicted scan quality visualization to acquire higher-quality points or fill the unscanned regions.

Z2P: Instant Visualization of Point Clouds

AbstractWe present a technique for visualizing point clouds using a neural network. Our technique allows for an instant preview of any point cloud, and bypasses the notoriously difficult surface reconstruction problem or the need to estimate oriented normals for splat‐based rendering. We cast the preview problem as a conditional image‐to‐image translation task, and design a neural network that translates point depth‐map directly into an image, where the point cloud is visualized as though a surface was reconstructed from it. Furthermore, the resulting appearance of the visualized point cloud can be, optionally, conditioned on simple control variables (e.g., color and light). We demonstrate that our technique instantly produces plausible images, and can, on‐the‐fly effectively handle noise, non‐uniform sampling, and thin surfaces sheets.

Open‐source terrestrial laser scanner for the virtualization of geometrical entities in AEC classrooms

AbstractThis paper depicts a case study that shows an open‐source Terrestrial Laser Scanner (TLS) for use on the virtualization of simple yet precise geometrical entities in AEC classrooms. For bringing this technology to AEC classrooms, an open‐source TLS was developed. The physical model was built using digital fabrication, its connectivity was established using accessible, affordable, open‐source sensors and actuators and its digital representation was developed using generative design tools. All elements together are synchronized in real‐time and the TLS becomes live, digitally twinned, geometry generator. The points that are generated by the TLS are gathered in a 3D virtual space in the form of virtual points. The present digitally twinned vision of laser scanning has thus three identified educational uses: (i) illustration of the measuring principle using open‐source hardware together with mathematics and statistics, (ii) illustration of the generation and visualization of point clouds in real‐time within a CAD environment, and (iii) perhaps with a vaster scope in AEC classrooms, illustration of the usage, analysis and identification of these point clouds. The development of this tool belongs to a vaster project for infusing Construction 4.0 technologies in AEC classrooms under development at the School of Civil Engineering of UPC‐BarcelonaTech. The illustration to students of all these concepts with accessible technological tools is expected to enlarge their vision of more advanced constructional technologies such as cyber‐physical systems for monitoring and surveying as well as digital technologies for reproducing “As‐Built” models.

HOLOGRAPHIC VISUALIZATION AND MANAGEMENT OF BIG POINT CLOUD

Abstract. The work aims to present and validate the workflow from the 3D survey to the visualization of cultural heritage objects using the innovative Euclideon Hologram Table©. Three case studies surveyed with three different systems and at three different scales have been selected: Santa Maria delle Grazie in Milan (terrestrial laser scanner for an architecturalscale), the village of Ghesc in the Ossola valley (UAV survey for an environmental scale) and the cuneiform clay tablet number 727 (structured light system for a detailscale). The whole process of transforming the 3D point/mesh model to hologram was verified, analysing the file formats, technical performance and specifications, file dimensions manageable, and details viewable. The first test shows great potentiality, becausethe hologram exploring is impressively fluid even when zooming to view a higher detail level, despite the high number of points/polygons. The power and performance of the point cloud 3D rendering engine result impressive. Nonetheless,different aspects need further research, from point cloud visualization quality to enhancing 3D model interaction.

A Deep Learning Method for Target Pose Estimation of Planetary Surface

The key technical challenge of 6D target attitude estimation from RGB-D image is to make full use of 2d and 3D image point cloud data. Previous traditional target pose estimation methods either extract information from RGB images and depth, respectively, or use expensive post-processing steps that limit their performance in highly cluttered scenarios and real-time applications. Therefore, in this paper, a method is proposed to estimate the 6D attitude of a group of known objects from RGB-D images. The network used is a heterogeneous architecture, which processes two data from different dimensions separately, and uses a new dense fusion network to extract pixel-level dense feature embedding, and estimate the target position and attitude from it. In addition, an end-to-end iterative attitude optimization method is integrated to further improve attitude estimation, while real-time visualization of point clouds and rendering of bounding boxes are realized. Experimental results show that this method is effective on open source data sets.

VISUALIZATION OF POINT CLOUD MODELS IN MOBILE AUGMENTED REALITY USING CONTINUOUS LEVEL OF DETAIL METHOD

Abstract. Point clouds have become one of the most popular sources of data in geospatial fields due to their availability and flexibility. However, because of the large amount of data and the limited resources of mobile devices, the use of point clouds in mobile Augmented Reality applications is still quite limited. Many current mobile AR applications of point clouds lack fluent interactions with users. In our paper, a cLoD (continuous level-of-detail) method is introduced to filter the number of points to be rendered considerably, together with an adaptive point size rendering strategy, thus improve the rendering performance and remove visual artifacts of mobile AR point cloud applications. Our method uses a cLoD model that has an ideal distribution over LoDs, with which can remove unnecessary points without sudden changes in density as present in the commonly used discrete level-of-detail approaches. Besides, camera position, orientation and distance from the camera to point cloud model is taken into consideration as well. With our method, good interactive visualization of point clouds can be realized in the mobile AR environment, with both nice visual quality and proper resource consumption.

Interactive dense point clouds in a game engine

With the development of 3D measurement systems, dense colored point clouds are increasingly available. However, up to now, their use in interactive applications has been restricted by the lack of support for point clouds in game engines. In addition, many of the existing applications for point clouds lack the capacity for fluent user interaction and application development. In this paper, we present the development and architecture of a game engine extension facilitating the interactive visualization of dense point clouds. The extension allows the development of game engine applications where users edit and interact with point clouds. To demonstrate the capabilities of the developed extension, a virtual reality head-mounted display is used and the rendering performance is evaluated. The result shows that the developed tools are sufficient for supporting real-time 3D visualization and interaction. Several promising use cases can be envisioned, including both the use of point clouds as 3D assets in interactive applications and leveraging the game engine point clouds in geomatics.

Spatiotemporal Camera-LiDAR Calibration: A Targetless and Structureless Approach

The demand for multimodal sensing systems for robotics is growing due to the increase in robustness, reliability and accuracy offered by these systems. These systems also need to be spatially and temporally co-registered to be effective. In this letter, we propose a targetless and structureless spatiotemporal camera-LiDAR calibration method. Our method combines a closed-form solution with a modified structureless bundle adjustment where the coarse-to-fine approach does not require an initial guess on the spatiotemporal parameters. Also, as 3D features (structure) are calculated from triangulation only, there is no need to have a calibration target or to match 2D features with the 3D point cloud which provides flexibility in the calibration process and sensor configuration. We demonstrate the accuracy and robustness of the proposed method through both simulation and real data experiments using multiple sensor payload configurations mounted to hand-held, aerial and legged robot systems. Also, qualitative results are given in the form of a colorized point cloud visualization.

Noise-robust transparent visualization of large-scale point clouds acquired by laser scanning

We propose a high-quality transparent visualization method suitable for large-scale laser-scanned point clouds. We call the method “stochastic point-based rendering (SPBR),” which is based on a novel stochastic algorithm. SPBR enables us to clearly observe the deep interior of laser-scanned 3D objects with the correct feeling of depth. The high quality of SPBR originates from the effect of “stochastic noise transparentization,” which is an effect to make the measurement noise transparent and invisible in the created images. We mathematically prove that this effect also makes the created transparent images coincide with the results of the conventional methods based on the alpha blending, which is time-consuming and impractical for large-scale laser-scanned point clouds. We also demonstrate the effectiveness of SPBR by applying it to modern buildings, cultural heritage objects, forests, and a factory. For all of the cases, the method works quite well, realizing clear and correct 3D see-through imaging of the laser-scanned objects.

A SERVICE-ORIENTED INDOOR POINT CLOUD PROCESSING PIPELINE

Abstract. Visualization of point clouds plays an important role in understanding the context of the digital representation of the built environment. Modern commodity mobile devices (e.g., smartphones and tablets), are capable of capturing representations in the form of 3D point clouds, with their depth-sensing and photogrammetry capabilities. Points clouds enable the encoding of important spatial and physical features of the built environment they represent. However, once captured, point clouds need to be processed before they can be used for further semantic enrichment and decision making. An integrated pipeline for such processes is crucial for use in larger and more complex enterprise systems and data analysis platforms, especially within the realm of Facility Management (FM) and Real Estate 4.0. We present and discuss a prototypical implementation for a service-oriented point cloud processing pipeline. The presented processing features focus on detecting and visualizing spatial deviations between as-is versus as-designed representations. We discuss the design and implementation of these processing features, and present experimental results. The presented approach can be used as a lightweight software component for processing indoor point clouds captured using commodity mobile devices, as well as primary deviation analysis, and also provides a processing link for further semantic enrichment of base-data for Building Information Modeling (BIM) and Digital Twin (DT) applications.

A new kind of atlas of zeolite building blocks.

We have analyzed structural motifs in the Deem database of hypothetical zeolites to investigate whether the structural diversity found in this database can be well-represented by classical descriptors, such as distances, angles, and ring sizes, or whether a more general representation of the atomic structure, furnished by the smooth overlap of atomic position (SOAP) method, is required to capture accurately structure-property relations. We assessed the quality of each descriptor by machine-learning the molar energy and volume for each hypothetical framework in the dataset. We have found that a SOAP representation with a cutoff length of 6 Å, which goes beyond near-neighbor tetrahedra, best describes the structural diversity in the Deem database by capturing relevant interatomic correlations. Kernel principal component analysis shows that SOAP maintains its superior performance even when reducing its dimensionality to those of the classical descriptors and that the first three kernel principal components capture the main variability in the dataset, allowing a 3D point cloud visualization of local environments in the Deem database. This "cloud atlas" of local environments was found to show good correlations with the contribution of a given motif to the density and stability of its parent framework. Local volume and energy maps constructed from the SOAP/machine learning analyses provide new images of zeolites that reveal smooth variations of local volumes and energies across a given framework and correlations between the contributions to volume and energy associated with each atom-centered environment.