Point Cloud Model Research Articles

Research on load monitoring technology of mine hoist based on machine vision

A hoist load monitoring method based on machine vision technology is proposed in this paper to address the frequent overloading accidents of mine hoists and the low safety and reliability of existing contact load monitoring technologies. The depth image of the skip undergoes time domain and spatial bilateral filtering algorithms for noise reduction, followed by conditional filtering and downsampling algorithms to remove redundant point cloud data. Point cloud recognition, extraction, segmentation, and alignment algorithms are then applied to quickly generate a skip point cloud model. A surface reconstruction optimization process combining greedy projection triangulation algorithm and void repair algorithm is proposed to obtain a smooth and complete sealing of the skip. The closed surface model volume is calculated using VTK volume function. Based on single-rope winding hoist, a load visual monitoring system is constructed for relevant experimental research. Results show that this method can accurately measure the loaded coal volume with a relative error range of 0.05%–4.13%, meeting practical application requirements while providing an effective way for non-contact accurate measurement of hoist loads in mines.

Telepresence augmentation for visual and haptic guided immersive teleoperation of industrial manipulator

Teleoperation under human guidance has become an effective solution to extend human’s reach in various environments. However, the teleoperation system still faces challenges of insufficient sense of both visual and haptic feedback from remote environments, which results in the inadequate guidance for the operator. In this paper, a visual/haptic integrated perception and reconstruction system (VHI-PRS) is developed to provide the operator with 3D visual information and effective haptic guidance. Specifically, a visual-based telepresence augmentation method is proposed to provide the operator with virtual-reality combined visual telepresence, where the real point cloud model is directly superimposed on virtual manipulator to avoid the time-consuming process of mesh model rendering. With the utilization of visual information, a haptic-based telepresence augmentation method is proposed to provide the operator with comprehensive force feedback, including the virtual guiding force, virtual repulsive force and real-time interactive force, which greatly helps reduce the workload of operator. Finally, a user study on grab-place task is carried out to verify the effectiveness of proposed system.

Improved pass-through dimensionality reduction for point cloud contour extraction

Obtaining a point cloud model that considers the overall information of large steel components as well as the local information of bolt holes is an important approach for the virtual assembly of large steel structures. A point cloud extraction method to improve the uniformity of the dimensionality reduction distribution is presented. Firstly, point cloud data characterising the overall information of a large-size component and local information of the bolt-hole group are obtained by combining vertical three-dimensional (3D) laser scanning and handheld 3D laser scanning. The point cloud is then triaxially equidistant and reduced in dimension based on improved straight-pass filtering and the corner point cloud is extracted using a planar point cloud distribution uniformity algorithm. Finally, the point cloud is restored to the same space to complete the contour extraction of the point cloud. The accuracy of the contour extraction method was verified by conducting point cloud feature extraction tests using standard components. Compared with conventional feature extraction, the method provides targeted local feature extraction for bars with a certain regularity of geometric configuration, reducing the time required for feature extraction and providing a brief database for the virtual assembly of steel joist beams.

Hierarchical SVM for Semantic Segmentation of 3D Point Clouds for Infrastructure Scenes

The incorporation of building information modeling (BIM) has brought about significant advancements in civil engineering, enhancing efficiency and sustainability across project life cycles. The utilization of advanced 3D point cloud technologies such as laser scanning extends the application of BIM, particularly in operations and maintenance, prompting the exploration of automated solutions for labor-intensive point cloud modeling. This paper presents a demonstration of supervised machine learning—specifically, a support vector machine—for the analysis and segmentation of 3D point clouds, which is a pivotal step in 3D modeling. The point cloud semantic segmentation workflow is extensively reviewed to encompass critical elements such as neighborhood selection, feature extraction, and feature selection, leading to the development of an optimized methodology for this process. Diverse strategies are implemented at each phase to enhance the overall workflow and ensure resilient results. The methodology is then evaluated using diverse datasets from infrastructure scenes of bridges and compared with state-of-the-art deep learning models. The findings highlight the effectiveness of supervised machine learning techniques at accurately segmenting 3D point clouds, outperforming deep learning models such as PointNet and PointNet++ with smaller training datasets. Through the implementation of advanced segmentation techniques, there is a partial reduction in the time required for 3D modeling of point clouds, thereby further enhancing the efficiency and effectiveness of the BIM process.

Usage of a Conventional Device with LiDAR Implementation for Mesh Model Creation

The trend of using conventional devices like mobile phones, tablets, and the other devices is gaining traction in improving customer service practices. This coincides with the growing popularity of building information modeling (BIM), which has led to increased exploration of various 3D object capture methods. Additionally, the technological boom has resulted in a surge of applications working with different 3D model formats including mesh models, point cloud, and TIN models. Among these, the usage of mesh models is experiencing particularly rapid growth. The main objective advantages of mesh models are their efficiency, scalability, flexibility, sense of detail, user-friendliness, and compatibility. The idea of this paper is to use a conventional device, specifically an iPad Pro equipped with light detection and ranging (LiDAR) technology, for creating mesh models. The different data capture methods employed by various applications will be compared to evaluate the final models´ precision. The accuracy of the 3D models generated by each application will be assessed by comparing the spatial coordinates of identical points distributed irregularly across the entire surface of the chosen object. Various available currently most-used applications were utilized in the process of data collection. In general, 3D representations of the object/area, etc., may be visualized, analyzed, and further processed in more formats such as TIN models, point cloud, or mesh models. Mesh models provide a visualization of the object mirroring the solid design of the real object, thus approximating reality in the closest way. This fact, along with automatized postprocessing after data acquisition, the ability to capture and visualize both convex and concave objects, and the possibility to use this type of 3D visualization for 3D printing, contribute to the decision to test and analyze mesh models. Consequently, the mesh models were created via the automatic post-processing, i.e., without external intervention. This fact leads to the problems of random coordinate systems being automatically pre-defined by every application. This research must deal with the resulting obstacles in order to provide a valid and credible comparative analysis. Various criteria may be applied to the mesh models’ comparisons, including objective qualitative and quantitative parameters and also the subjective ones. The idea of this research is not to analyze the data acquisition process in detail, but instead to assess the possibilities of the applications for the basic users.

Comparison of point cloud and image-based models for calorimeter fast simulation

Score based generative models are a new class of generative models that have been shown to accurately generate high dimensional calorimeter datasets. Recent advances in generative models have used images with 3D voxels to represent and model complex calorimeter showers. Point clouds, however, are likely a more natural representation of calorimeter showers, particularly in calorimeters with high granularity. Point clouds preserve all of the information of the original simulation, more naturally deal with sparse datasets, and can be implemented with more compact models and data files. In this work, two state-of-the-art score based models are trained on the same set of calorimeter simulation and directly compared.

Reactive UAV-based automatic tunnel surface defect inspection with a field test

This work addresses the problem of automatic tunnel surface defect inspection using unmanned aerial vehicles (UAVs). The research aims at proposing a robust and efficient monitoring method for image data acquisition and processing in complex and dark tunnel environments. A method, called Proximity Move-Pause-Photo for Surface Defect Inspection (PMPP-SDI), is proposed by combining reactive flying control strategies with a grid scanning pattern to capture high-quality image data from multiple views and angles. The image data is then used to generate a 3D point cloud model of the tunnel surface for structural condition assessment. The method is tested in a field experiment in a railway tunnel in Ireland, and the results show that it can achieve stable navigation, high-resolution reconstruction, and accurate defect detection. The paper discusses the advantages and limitations of the method, and suggests improving the control/navigation intelligence, data quality, and defect analysis as the future research directions.

Dip-NeRF: Depth-Based Anti-Aliased Neural Radiance Fields

Neural radiation field (NeRF)-based novel view synthesis methods are gaining popularity for their ability to generate detailed and realistic images. However, most NeRF-based methods only use images to learn scene representations, ignoring the importance of depth information. The Zip-NeRF method has achieved impressive results in unbounded scenes by combining anti-aliasing techniques and mesh representations. However, the method requires a large number of input images and may perform poorly in complex scenes. Our method incorporates the advantages of Zip-NeRF and incorporates depth information to reduce the number of required images and solve the scale-free problem in borderless scenes. Experimental results show that our method effectively reduces the training time.And we can generate high-quality images and fine point cloud models using few images, even in complex scenes with numerous occlusions.

Integrating Drone Imagery and AI for Improved Construction Site Management through Building Information Modeling

In the rapidly advancing field of construction, digital site management and Building Information Modeling (BIM) are pivotal. This study explores the integration of drone imagery into the digital construction site management process, aiming to create BIM models with enhanced object recognition capabilities. Initially, the research sought to achieve photorealistic rendering of point cloud models (PCMs) using blur/sharpen filters and generative adversarial network (GAN) models. However, these techniques did not fully meet the desired outcomes for photorealistic rendering. The research then shifted to investigating additional methods, such as fine-tuning object recognition algorithms with real-world datasets, to improve object recognition accuracy. The study’s findings present a nuanced understanding of the limitations and potential pathways for achieving photorealistic rendering in PCM, underscoring the complexity of the task and laying the groundwork for future innovations in this area. Although the study faced challenges in attaining the original goal of photorealistic rendering for object detection, it contributes valuable insights that may inform future research and technological development in digital construction site management.

Interpretable3D: An Ad-Hoc Interpretable Classifier for 3D Point Clouds

3D decision-critical tasks urgently require research on explanations to ensure system reliability and transparency. Extensive explanatory research has been conducted on 2D images, but there is a lack in the 3D field. Furthermore, the existing explanations for 3D models are post-hoc and can be misleading, as they separate explanations from the original model. To address these issues, we propose an ad-hoc interpretable classifier for 3D point clouds (i.e., Interpretable3D). As an intuitive case-based classifier, Interpretable3D can provide reliable ad-hoc explanations without any embarrassing nuances. It allows users to understand how queries are embedded within past observations in prototype sets. Interpretable3D has two iterative training steps: 1) updating one prototype with the mean of the embeddings within the same sub-class in Prototype Estimation, and 2) penalizing or rewarding the estimated prototypes in Prototype Optimization. The mean of embeddings has a clear statistical meaning, i.e., class sub-centers. Moreover, we update prototypes with their most similar observations in the last few epochs. Finally, Interpretable3D classifies new samples according to prototypes. We evaluate the performance of Interpretable3D on four popular point cloud models: DGCNN, PointNet2, PointMLP, and PointNeXt. Our Interpretable3D demonstrates comparable or superior performance compared to softmax-based black-box models in the tasks of 3D shape classification and part segmentation. Our code is released at: github.com/FengZicai/Interpretable3D.

Frozen CLIP Transformer Is an Efficient Point Cloud Encoder

The pretrain-finetune paradigm has achieved great success in NLP and 2D image fields because of the high-quality representation ability and transferability of their pretrained models. However, pretraining such a strong model is difficult in the 3D point cloud field due to the limited amount of point cloud sequences. This paper introduces Efficient Point Cloud Learning (EPCL), an effective and efficient point cloud learner for directly training high-quality point cloud models with a frozen CLIP transformer. Our EPCL connects the 2D and 3D modalities by semantically aligning the image features and point cloud features without paired 2D-3D data. Specifically, the input point cloud is divided into a series of local patches, which are converted to token embeddings by the designed point cloud tokenizer. These token embeddings are concatenated with a task token and fed into the frozen CLIP transformer to learn point cloud representation. The intuition is that the proposed point cloud tokenizer projects the input point cloud into a unified token space that is similar to the 2D images. Comprehensive experiments on 3D detection, semantic segmentation, classification and few-shot learning demonstrate that the CLIP transformer can serve as an efficient point cloud encoder and our method achieves promising performance on both indoor and outdoor benchmarks. In particular, performance gains brought by our EPCL are 19.7 AP50 on ScanNet V2 detection, 4.4 mIoU on S3DIS segmentation and 1.2 mIoU on SemanticKITTI segmentation compared to contemporary pretrained models. Code is available at \url{https://github.com/XiaoshuiHuang/EPCL}.

Reinforcement Learning-Based Simulation of Seal Engraving Robot in the Context of Artificial Intelligence

The rapid development of robotics technology has made people's lives and work more convenient and efficient. The research and simulation of robots combined with reinforcement learning intelligent algorithms have become a hotspot in various fields of robot applications. In view of this, this study is based on deep reinforcement learning convolutional neural networks, combined with point cloud models, proximal strategy optimization algorithms, and flexible action evaluation algorithms. A seal cutting robot based on deep reinforcement learning has been proposed. The final results show that the descent speed of the seal cutting robot with the root mean square difference as the performance standard is about 1% faster than the flexible action evaluation algorithm. About 2% faster than the proximal strategy optimization algorithm. It is about 4% faster than the deep deterministic strategy gradient algorithm. This indicates that the research model has certain advantages in terms of actual accuracy after cutting. The fluctuation of this model is about 10% smaller than the evaluation of flexible actions and about 60% smaller than the gradient of deep deterministic strategies. Therefore, the research model has the highest overall stability without falling into local optima. In addition, compared to the near end strategy optimization algorithm, it falls into local optima, resulting in a low coincidence degree of about 17%. The deep deterministic strategy gradient algorithm has a large fluctuation amplitude during the seal cutting process, and the overall curve is relatively slow, with a final overlap of about 70%. The overlap degree of flexible action evaluation is slightly higher by about 83%. The maximum stability of the model's overlap is best around 90%. Through experiments, it can be found that the seal cutting robot proposed in the study based on deep reinforcement learning maintains certain advantages in performance indicators in various types of tests.

Evaluating the Point Cloud of Individual Trees Generated from Images Based on Neural Radiance Fields (NeRF) Method

Three-dimensional (3D) reconstruction of trees has always been a key task in precision forestry management and research. Due to the complex branch morphological structure of trees themselves and the occlusions from tree stems, branches and foliage, it is difficult to recreate a complete three-dimensional tree model from a two-dimensional image by conventional photogrammetric methods. In this study, based on tree images collected by various cameras in different ways, the Neural Radiance Fields (NeRF) method was used for individual tree dense reconstruction and the exported point cloud models are compared with point clouds derived from photogrammetric reconstruction and laser scanning methods. The results show that the NeRF method performs well in individual tree 3D reconstruction, as it has a higher successful reconstruction rate, better reconstruction in the canopy area and requires less images as input. Compared with the photogrammetric dense reconstruction method, NeRF has significant advantages in reconstruction efficiency and is adaptable to complex scenes, but the generated point cloud tend to be noisy and of low resolution. The accuracy of tree structural parameters (tree height and diameter at breast height) extracted from the photogrammetric point cloud is still higher than those derived from the NeRF point cloud. The results of this study illustrate the great potential of the NeRF method for individual tree reconstruction, and it provides new ideas and research directions for 3D reconstruction and visualization of complex forest scenes.

Data model-based toolpath generation techniques for CNC milling machines

Introduction: With the development of computer technology and data modeling, the use of point cloud models to generate tool paths is particularly important for improving productivity and accuracy.Methods: This study proposes a new method that first preprocesses the point cloud data using four-point denoising and octree methods to improve processing efficiency. Subsequently, roughing tool paths were analyzed using the layer slicing method and finishing paths using the residual height method.Results and Discussion: The experimental results show that the layer slicing method has a minimum error close to 10% on the roughing path generation and the computation time is reduced to 35 s, while the residual height method has an error rate of 10.17% on the finishing path and the computation time is only 11.82 s, which reflects a high trajectory smoothness and accuracy. The above results show that the study not only optimizes the tool path generation process and improves the machining efficiency and accuracy, but also demonstrates the potential application of point cloud models in the machining of complex parts.Conclusion: The novel tool roughing and finishing methods provide more reliable path planning for actual machining operations, and future research will be devoted to further improving the performance of the data processing algorithms and exploring more efficient path planning strategies to facilitate automated production.

Rapid pre-typhoon safety inspection system for improving construction site resilience

Coastal cities are vulnerable to typhoon disasters due to their unique geographical location. Although significant progress has been made in the safety early warning system for construction sites, there are few studies on site safety management under typhoon disasters. Moreover, traditional manual methods of on-site emergency management are lagging behind and rigid, making it difficult to respond quickly to safety inspections manually. This study aims to automatically identify safety hazards and provide effective response measures during typhoon warnings, ultimately enhancing the safety of construction sites. Therefore, based on the MATLAB platform, this study developed a rapid safety inspection system (RSS-Typh) for the construction site during the typhoon warning period. In addition, the standard operating procedures (SOP) related to the system are formulated to assist the comprehensive safety inspection of the site before the typhoon and improve the inspection efficiency. The specific operation can be divided into three steps: 1.Using a combination of drones and handheld laser scanners to complete the complete construction site data acquisition in less than 2 h, and complete the modeling and refinement of the point cloud model in 5 h; 2.Using the developed hidden danger detection algorithm based on automatic point cloud to effectively identify the deformation-related hidden dangers at the construction site; 3.Use the designed automatic query security countermeasure GUI system to make rapid rectification. This study assists site managers in swiftly implementing effective safety management on construction sites, elevating the safety level of the site before typhoon. Furthermore, it serves to advance the automation of safety management in construction sites, promoting further developments in this field.

Mono‐MVS: textureless‐aware multi‐view stereo assisted by monocular prediction

AbstractThe learning‐based multi‐view stereo (MVS) methods have made remarkable progress in recent years. However, these methods exhibit limited robustness when faced with occlusion, weak or repetitive texture regions in the image. These factors often lead to holes in the final point cloud model due to excessive pixel‐matching errors. To address these challenges, we propose a novel MVS network assisted by monocular prediction for 3D reconstruction. Our approach combines the strengths of both monocular and multi‐view branches, leveraging the internal semantic information extracted from a single image through monocular prediction, along with the strict geometric relationships between multiple images. Moreover, we adopt a coarse‐to‐fine strategy to gradually reduce the number of assumed depth planes and minimise the interval between them as the resolution of the input images increases during the network iteration. This strategy can achieve a balance between the computational resource consumption and the effectiveness of the model. Experiments on the DTU, Tanks and Temples, and BlendedMVS datasets demonstrate that our method achieves outstanding results, particularly in textureless regions.

RANSAC Algorithm and Distributed Framework for Point Cloud Processing of Ancient Buildings

This paper introduces a comprehensive framework for the point cloud processing of traditional buildings. The framework includes segmentation using the RANSAC algorithm and distributed storage based on a fuzzy weighting approach. The methodology employs a height threshold parameter for segmentation to extract building structural elements effectively. Furthermore, an interactive access control model distributes tasks across nodes to achieve load balancing during point cloud matching and analysis. Experimental results demonstrate segmentation accuracy exceeding 99% and alignment time reduction to 967 ms for point cloud models. The distributed computation efficiency reaches 0.8, outperforming conventional methods. The proposed techniques enable accurate dimensional capture, efficient data storage, and information extraction from traditional buildings to support digital preservation.

Research on Intelligent Design of Geometric Factor Encoding for Aircraft Engine Turbine Structures

In recent years, with the rapid development of computer technology and artificial intelligence design technology, multiple possible design solutions can be quickly generated by transforming the experience and knowledge of structural design into computer executable rules and algorithms. To achieve intelligent design of aircraft engines, this paper proposes an encoding model for the turbine rotor structure of aircraft engines using geometric encoding technology. The turbine rotor structure of aircraft engines is divided into several units according to geometric similarity types, these units continue to be divided into attribute sets according to their functional types, connection relationships, and material properties. These attribute sets can be encoded using geometric encoding technology. The experiment simulated that these codes, for the point cloud modeling of turbine rotor structure, can be quickly achieved and they combine various algorithms to display the point cloud model of the turbine rotor in the Microsoft Visual studio MFC class library. The results show that by creating geometric codes for the turbine rotor of aircraft engines, it is possible to quickly create and display point cloud models of the turbine rotor structure, laying the foundation for subsequent application of machine learning to solve and find the optimal design solution.

DCCN: A dual-cross contrastive neural network for 3D point cloud representation learning

The proliferation of depth cameras and LiDAR sensors in actual industrial environments has fueled the pursuit of an effective and efficient 3D point cloud model that enables us to perceive and interact with the physical world. However, the intrinsic complexity of 3D semantic information poses significant challenges to model design, including spatial rotation invariance and irregular point cloud structure, which fundamentally impact the representation and behavior of 3D point cloud systems. Existing have either heavily relied on labeling information in a supervised learning setting or failed to effectively capture the inherent patterns of the 3D point clouds within a self-supervised learning framework, leading to poor performance in specific downstream tasks. To address these limitations, this paper introduces a self-supervised framework, Dual-Cross Contrastive Neural Network (DCCN) for 3D point cloud self-supervised representation learning. DCCN leverages cross-view, cross-network, and domain-specific knowledge distillation to enhance the extraction of hidden features from point clouds and fully exploit the capabilities of the encoder. Our DCCN employs a pseudo-Siamese network consisting of an online network and a target network, facilitating knowledge interaction and distillation. The method extracts internal states from augmented 3D point cloud by learning cross-view relationships and optimizes model parameters through intra-modal cross-network learning. We incorporate a momentum-updating mechanism without shared weights in the Siamese network architecture to distill knowledge and enhance the role differentiation the online and target networks. Experimental results demonstrate that our approach outperforms a range of supervised and self-supervised learning methods across a series of downstream tasks consisting of four tasks in three representative datasets. Ablation studies validate the component-wise effectiveness of cross-view, cross-network, and moment-updating learning objectives in achieving superior point cloud representation. The overall findings establish our method, DCCN, as an effective solution for 3D point cloud representation learning in real-world applications.

Harnessing Digital Twins for Agriculture 5.0: A Comparative Analysis of 3D Point Cloud Tools

Digital twins are essential in Agriculture 5.0, providing an accurate digital representation of agricultural objects and processes, enabling data-driven decision-making, the simulation of future scenarios, and innovation for a more efficient and sustainable agriculture. The main objective of this article is to review and compare the main tools for the development of digital twins for Agriculture 5.0 applications using 3D point cloud models created from photogrammetry techniques. For this purpose, the most commonly used tools for the development of these 3D models are presented. As a methodological approach, a qualitative comparison of the main characteristics of these tools was carried out. Then, based on some images taken in an orange grove, a quality analysis of the 3D point cloud models obtained by each of the analyzed tools was carried out. We also obtained a synthetic quality index in order to have a way to categorize the different pieces of software. Finally, as a conclusion, we compared the performance of the different software tools and the point clouds obtained by considering objective metrics (from the 3D quality assessment) and qualitative metrics in the synthetic quality index. With this index, we found that OpenDroneMap was the best software in terms of quality-cost ratio. Also, the paper introduces the concept of Agriculture 6.0, exploring the integration of advancements from Agriculture 5.0 to envision the potential evolution of agricultural practices and technologies, considering their impact on social and economic aspects.