Point Cloud Segmentation Method Research Articles

A Point Cloud Segmentation Method for Pigs from Complex Point Cloud Environments Based on the Improved PointNet++

In animal husbandry applications, segmenting live pigs in complex farming environments faces many challenges, such as when pigs lick railings and defecate within the acquisition environment. The pig’s behavior makes point cloud segmentation more complex because dynamic animal behaviors and environmental changes must be considered. This further requires point cloud segmentation algorithms to improve the feature capture capability. In order to tackle the challenges associated with accurately segmenting point cloud data collected in complex real-world scenarios, such as pig occlusion and posture changes, this study utilizes PointNet++. The SoftPool pooling method is employed to implement a PointNet++ model that can achieve accurate point cloud segmentation for live pigs in complex environments. Firstly, the PointNet++ model is modified to make it more suitable for pigs by adjusting its parameters related to feature extraction and sensory fields. Then, the model’s ability to capture the details of point cloud features is further improved by using SoftPool as the point cloud feature pooling method. Finally, registration, filtering, and extraction are used to preprocess the point clouds before integrating them into a dataset for manual annotation. The improved PointNet++ model’s segmentation ability was validated and redefined with the pig point cloud dataset. Through experiments, it was shown that the improved model has better learning ability across 529 pig point cloud data sets. The optimal mean Intersection over Union (mIoU) was recorded at 96.52% and the accuracy at 98.33%. This study has achieved the automatic segmentation of highly overlapping pigs and pen point clouds. This advancement enables future animal husbandry applications, such as estimating body weight and size based on 3D point clouds.

Design and application of data security privacy protection technology in dance action sequence arrangement and display system

In view of the excessive consumption of difficult resources in traditional dance choreography, insufficient data in choreography and data leakage crisis in the system, this paper puts forward artificial intelligence technology to study the protection of dance choreography and system data security and privacy. In this paper, the directed graph neural network is used to intelligently arrange the dance movements, and the Hidden Markov Model (HMM) is used to process the dance scene data so that the output data is highly similar to the original data. In the protection of data security and privacy, the method of data encryption is used, and the encrypted data needs to be decrypted by algorithm for reading. The experimental results show that the more complete the dance movement display, the higher the dance recognition rate, and the recognition rate of the dynamic light projection algorithm is greater than 90 % regardless of several sets of dance movements, while the recognition rate of the traditional light projection algorithm is not up to the standard regardless of several sets of movements, and the recognition rate of the point cloud segmentation method is higher than 90 % only in 3–4 sets of dance and 5–6 sets of dance movements.

Multi-Modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation

3D point cloud semantic segmentation has a wide range of applications. Recently, weakly supervised point cloud segmentation methods have been proposed, aiming to alleviate the expensive and laborious manual annotation process by leveraging scene-level labels. However, these methods have not effectively exploited the rich geometric information (such as shape and scale) and appearance information (such as color and texture) present in RGB-D scans. Furthermore, current approaches fail to fully leverage the point affinity that can be inferred from the feature extraction network, which is crucial for learning from weak scene-level labels. Additionally, previous work overlooks the detrimental effects of the long-tailed distribution of point cloud data in weakly supervised 3D semantic segmentation. To this end, this paper proposes a simple yet effective scene-level weakly supervised point cloud segmentation method with a newly introduced multi-modality point affinity inference module. The point affinity proposed in this paper is characterized by features from multiple modalities (e.g., point cloud and RGB), and is further refined by normalizing the classifier weights to alleviate the detrimental effects of long-tailed distribution without the need of the prior of category distribution. Extensive experiments on the ScanNet and S3DIS benchmarks verify the effectiveness of our proposed method, which outperforms the state-of-the-art by ~4% to ~ 6% mIoU. Codes are released at https://github.com/Sunny599/AAAI24-3DWSSG-MMA.

A Point Cloud Segmentation Method for Dim and Cluttered Underground Tunnel Scenes Based on the Segment Anything Model

In recent years, point cloud segmentation technology has increasingly played a pivotal role in tunnel construction and maintenance. Currently, traditional methods for segmenting point clouds in tunnel scenes often rely on a multitude of attribute information, including spatial distribution, color, normal vectors, intensity, and density. However, the underground tunnel scenes show greater complexity than road tunnel scenes, such as dim light, indistinct boundaries of tunnel walls, and disordered pipelines. Furthermore, issues pertaining to data quality, such as the lack of color information and insufficient annotated data, contribute to the subpar performance of conventional point cloud segmentation algorithms. To address this issue, a 3D point cloud segmentation framework specifically for underground tunnels is proposed based on the Segment Anything Model (SAM). This framework effectively leverages the generalization capability of the visual foundation model to automatically adapt to various scenes and perform efficient segmentation of tunnel point clouds. Specifically, the tunnel is first sliced along its direction on the tunnel line. Then, each sliced point cloud is projected onto a two-dimensional plane. Various projection methods and point cloud coloring techniques are employed to enhance SAM’s segmentation performance in images. Finally, the semantic segmentation of the entire underground tunnel is achieved by a small set of manually annotated semantic labels used as prompts in a progressive and recursive manner. The key feature of this method lies in its independence from model training, as it directly and efficiently addresses tunnel point cloud segmentation challenges by capitalizing on the generalization capability of foundation model. Comparative experiments against classical region growing algorithms and PointNet++ deep learning algorithms demonstrate the superior performance of our proposed algorithm.

Defect segmentation with local embedding in industrial 3D point clouds based on transformer

Three-dimensional (3D) defect detection provides an effective method for improving industrial production efficiency. However, the 3D dataset is scarce, which is valuable for the industrial production field. This study proposes a new approach for detecting defect point clouds, which can provide an end-to-end 3D defect detection model. A self-attention mechanism is used to enrich the semantic relationships between local neighborhood features and global features based on the connection between them. Through adding multi-channel features, the rich structural features of the target point cloud are obtained, and the defect areas are accurately segmented to finally complete the 3D point cloud defect detection task. Furthermore, the multi-feature fusion in the model makes the segmented defect regions closer to the ground truth. Our method outperforms four state-of-the-art point cloud segmentation methods in terms of both segmentation region accuracy and defect detection point cloud accuracy. In the field of 3D defect detection, it provides an effective method to detect 3D information of industrial products.

IEFM and IDS: Enhancing 3D environment perception via information encoding in indoor point cloud semantic segmentation

The point cloud semantic segmentation plays a significant role in the understanding of 3D environment. However, current 3D point cloud segmentation methods pay little attention in the inevitable blurring and loss of point information within deep network, which considerably impairs the segmentation performance, particularly in the complex and colorful indoor scenes. To overcome the inevitable loss of point information (position, color and normal vector), in this paper, we propose a brand-new Information Encoding and Fusion Module (IEFM), comprising a point Information Encoding Method (IEM) and an optimized Multi-Encoding Fusion Method (EFM). In terms of constructing the innovative bias standardization of point information and effectively merging other information encodings into position encoding, IEFM adaptively complements the loss of the point information and thereby achieving enhanced environment perception ability. Additionally, although the proposed IEFM is capable of managing multi-class point information, the encoding interference resulting from the coexistence of multiple information is quite unavoidable, leading to detrimental consequences for the overall semantic segmentation performance. Therefore, to reduce the information encoding interference problem, we further innovatively propose the Information Distribution Strategy (IDS), in terms of hierarchically distribute kinds of point information, so that the interference of multiple information encodings will be extremely mitigated and achieving more accurate indoor point cloud semantic segmentation. Benefiting from the modular design, the proposed IEFM and IDS can be easily inserted in existing point-based point cloud segmentation models. The experimental results have shown the effectiveness of our proposed methods across multiple state-of-the-art models and benchmarks (S3DIS and ScanNet), achieving competitive performance of 77.4% mIoU on the large-scale S3DIS benchmark.

Improved 3D point cloud segmentation for accurate phenotypic analysis of cabbage plants using deep learning and clustering algorithms

Plant phenotyping is essential for understanding and managing plant growth and development. 3D point clouds provide a better understanding of plant 3D structures. Point cloud segmentation is the basis for studying the 3D structure of plants through 3D point clouds, and accurate point cloud segmentation is crucial for extracting relevant phenotypic parameters. In this study, cabbage was used as an example, and a plant point cloud segmentation method combining deep learning algorithms and clustering algorithms was proposed. Specifically, a cabbage point cloud dataset was constructed using a 3D scanning platform. The ASAP attention module was incorporated into the PointNet++ model, resulting in the improved ASAP-PointNet model. Superior semantic segmentation performance on the cabbage point cloud dataset was demonstrated by this model. The workflow of the DBSCAN algorithm was also optimized, which exhibited enhanced performance in organ-level plant point cloud segmentation experiments. Subsequently, five phenotypic features were extracted. The experimental results revealed that an accuracy of 0.95 and an intersection over union (IoU) of 0.86 for semantic segmentation were achieved by the ASAP-PointNet model. The correlation coefficients between the four phenotype parameters (plant height, leaf length, leaf width, and leaf area) and their corresponding measured values were 0.96, 0.91, 0.95, and 0.94, respectively. An automated data analysis, from plant 3D point clouds to phenotypic parameters, is enabled by the proposed method, which serves as a valuable reference for plant phenotype research.

Segmentation Clustering Algorithm for Point Cloud Processing of Targets on The Ground

After the ground extraction algorithm, there are multiple objects, including the target in the non-ground point cloud. To effectively extract the features of individual objects and overcome the dependence of the traditional algorithm based on the normal vector and curvature of the region growing point cloud segmentation method on the initial point, the normal vector angle in the neighborhood, and the curvature threshold selection. In this paper, we propose a fast clustering segmentation method for region-growing point clouds based on neighborhood density. The test results show that the point cloud segmentation algorithm proposed in this paper has better performance than the traditional Euclidean clustering point cloud segmentation algorithm and region growing point cloud segmentation algorithm in terms of timeliness and correct segmentation rate. This new segmentation clustering algorithm can quickly implement automatic point cloud clustering segmentation of ground objects.

3D reconstruction and defect pattern recognition of bonding wire based on stereo vision

AbstractNon‐destructive detection of wire bonding defects in integrated circuits (IC) is critical for ensuring product quality after packaging. Image‐processing‐based methods do not provide a detailed evaluation of the three‐dimensional defects of the bonding wire. Therefore, a method of 3D reconstruction and pattern recognition of wire defects based on stereo vision, which can achieve non‐destructive detection of bonding wire defects is proposed. The contour features of bonding wires and other electronic components in the depth image is analysed to complete the 3D reconstruction of the bonding wires. Especially to filter the noisy point cloud and obtain an accurate point cloud of the bonding wire surface, a point cloud segmentation method based on spatial surface feature detection (SFD) was proposed. SFD can extract more distinct features from the bonding wire surface during the point cloud segmentation process. Furthermore, in the defect detection process, a directional discretisation descriptor with multiple local normal vectors is designed for defect pattern recognition of bonding wires. The descriptor combines local and global features of wire and can describe the spatial variation trends and structural features of wires. The experimental results show that the method can complete the 3D reconstruction and defect pattern recognition of bonding wires, and the average accuracy of defect recognition is 96.47%, which meets the production requirements of bonding wire defect detection.

Study on Single-Tree Extraction Method for Complex RGB Point Cloud Scenes

With the development of sensor technology and point cloud generation techniques, there has been an increasing amount of high-quality forest RGB point cloud data. However, popular clustering-based point cloud segmentation methods are usually only suitable for pure forest scenes and not ideal for scenes with multiple ground features or complex terrain. Therefore, this study proposes a single-tree point cloud extraction method that combines deep semantic segmentation and clustering. This method first uses a deep semantic segmentation network, Improved-RandLA-Net, which is developed based on RandLA-Net, to extract point clouds of specified tree species by adding an attention chain to improve the model’s ability to extract channel and spatial features. Subsequently, clustering is employed to extract single-tree point clouds from the segmented point clouds. The feasibility of the proposed method was verified in the Gingko site, the Lin’an Pecan site, and a Fraxinus excelsior site in a conference center. Finally, semantic segmentation was performed on three sample areas using pre- and postimproved RandLA-Net. The experiments demonstrate that Improved-RandLA-Net had significant improvements in Accuracy, Precision, Recall, and F1 score. At the same time, based on the semantic segmentation results of Improved-RandLA-Net, single-tree point clouds of three sample areas were extracted, and the final single-tree recognition rates for each sample area were 89.80%, 75.00%, and 95.39%, respectively. The results demonstrate that our proposed method can effectively extract single-tree point clouds in complex scenes.

Three-Dimensional Lidar Localization and Mapping with Loop-Closure Detection Based on Dense Depth Information

This paper presents a novel lidar SLAM system for localizing a mobile robot to build a map of the environment. To identify the unknown transform matrix, we design a new scan-matching approach, in which a point cloud segmentation algorithm is additionally integrated. Different from the traditional normal distribution transform algorithm for point cloud registration, our newly proposed one additionally incorporates a ground point remover and a point cloud segmentation method. By employing the point cloud segmentation algorithm to divide the point cloud space into different cells, the newly proposed algorithm can guarantee the continuity and convergence of the cost function. To tackle the recognition difficulties that the camera-based loop-closure detection heavily depends on the environment’s appearance, a depth-completion algorithm is introduced to fuse sensor data to ensure the robustness of the algorithm. Moreover, the bags of binary words (DBoW) are adopted to improve the image-matching quality. Finally, experimental results are presented to illustrate the effectiveness of the proposed system.

A point cloud segmentation method for power lines and towers based on a combination of multiscale density features and point-based deep learning

ABSTRACT The point segmentation of power lines and towers aims to use unmanned aerial vehicles (UAVs) for the inspection of power facilities, risk detection and modelling. Because of the unclear spatial relationship between the point clouds, the point segmentation of power lines and towers is challenging. In this paper, the power line and tower point datasets are constructed using Light Detection and Ranging (LiDAR) and a point segmentation method is proposed based on multiscale density features and a point-based deep learning network. First, the data are blocked and the neighbourhood is constructed. Second, the point clouds are downsampled to produce sparse point clouds. The point clouds before and after sampling are rotated, and their density is calculated. Next, a direct mapping method is selected to fuse the density information; a lightweight network is built to learn the features. Finally, the point clouds are segmented by concatenating the local features provided by PointCNN. The algorithm performs effectively on different types of power lines and towers. The mean interaction over union is 82.73%, and the overall accuracy can reach 91.76%. This approach can achieve the end-to-end integration of segmentation and provide theoretical support for the segmentation of large scenic point clouds.

Point Cloud Deep Learning Network Based on Balanced Sampling and Hybrid Pooling

The automatic semantic segmentation of point cloud data is important for applications in the fields of machine vision, virtual reality, and smart cities. The processing capability of the point cloud segmentation method with PointNet++ as the baseline needs to be improved for extremely imbalanced point cloud scenes. To address this problem, in this study, we designed a weighted sampling method based on farthest point sampling (FPS), which adjusts the sampling weight value according to the loss value of the model to equalize the sampling process. We also introduced the relational learning of the neighborhood space of the sampling center point in the feature encoding process, where the feature importance is distinguished by using a self-attention model. Finally, the global-local features were aggregated and transmitted using the hybrid pooling method. The experimental results of the six-fold crossover experiment showed that on the S3DIS semantic segmentation dataset, the proposed network achieved 9.5% and 11.6% improvement in overall point-wise accuracy (OA) and mean of class-wise intersection over union (MIoU), respectively, compared with the baseline. On the Vaihingen dataset, the proposed network achieved 4.2% and 3.9% improvement in OA and MIoU, respectively, compared with the baseline. Compared with the segmentation results of other network models on public datasets, our algorithm achieves a good balance between OA and MIoU.

PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage

Segmentation of plant point clouds to obtain high-precise morphological traits is essential for plant phenotyping. Although the fast development of deep learning has boosted much research on segmentation of plant point clouds, previous studies mainly focus on the hard voxelization-based or down-sampling-based methods, which are limited to segmenting simple plant organs. Segmentation of complex plant point clouds with a high spatial resolution still remains challenging. In this study, we proposed a deep learning network plant segmentation transformer (PST) to achieve the semantic segmentation of rapeseed plants point clouds acquired by handheld laser scanning (HLS) with the high spatial resolution, which can characterize the tiny siliques as the main traits targeted. PST is composed of: (i) a dynamic voxel feature encoder (DVFE) to aggregate the point features with the raw spatial resolution; (ii) the dual window sets attention blocks to capture the contextual information; and (iii) a dense feature propagation module to obtain the final dense point feature map. We then integrated PST with an instance segmentation head in the point grouping network (PointGroup) and developed PST-PointGroup (PG) to achieve the instance segmentation of the siliques. The results proved that PST and PST-PG achieved superior performance in semantic and instance segmentation tasks. For the semantic segmentation, the mean IoU, mean Precision, mean Recall, mean F1-score, and overall accuracy of PST were 93.96%, 97.29%, 96.52%, 96.88%, and 97.07%, achieving an improvement of 7.62, 3.28, 4.8, 4.25, and 3.88 percentage points compared to the second-best state-of-the-art network position adaptive convolution (PAConv). For instance segmentation, PST-PG reached 89.51%, 89.85%, 88.83% and 82.53% in mCov, mWCov, mPerc90, and mRec90, achieving an improvement of 2.93, 2.21, 1.99, and 5.9 percentage points compared to the original instance segmentation network PointGroup. This study extends the phenotyping of rapeseed plants in an end-to-end way and proves that the deep-learning-based point cloud segmentation method has a great potential for resolving dense plant point clouds with complex morphological traits.

A Method for the Automatic Extraction of Support Devices in an Overhead Catenary System Based on MLS Point Clouds

A mobile laser scanning (MLS) system can acquire railway scene information quickly and provide a data foundation for regular railway inspections. The location of the catenary support device in an electrified railway system has a direct impact on the regular operation of the power supply system. However, multi-type support device data accounts for a tiny proportion of the whole railway scene, resulting in its poor characteristic expression in the scene. Therefore, using traditional point cloud filtering or point cloud segmentation methods alone makes it difficult to achieve an effective segmentation and extraction of the support device. As a result, this paper proposes an automatic extraction algorithm for complex railway support devices based on MLS point clouds. First, the algorithm obtains hierarchies of the pillar point clouds and the support device point clouds in the railway scene through high stratification and then realizes the noise that was point-cloud-filtered in the scene. Then, the center point of the pillar device is retrieved from the pillar corridor by a neighborhood search, and then the locating and initial extracting of the support device are realized based on the relatively stable spatial topological relationship between the pillar and the support device. Finally, a post-processing optimization method integrating the pillar filter and the voxelized projection filter is designed to achieve the accurate and efficient extraction of the support device based on the feature differences between the support device and other devices in the initial extraction results. Furthermore, in the experimental part, we evaluate the treatment effect of the algorithm in six types of support devices, three types of support device distribution scenes, and two types of railway units. The experimental results show that the average extraction IoU of the multi-type support device, support device distribution scenes, and railway unit were 97.20%, 94.29%, and 96.11%, respectively. In general, the proposed algorithm can achieve the accurate and efficient extraction of various support devices in different scenes, and the influence of the algorithm parameters on the extraction accuracy and efficiency is elaborated in the discussion section.

POINT CLOUD SLICING-BASED EXTRACTION OF INDOOR COMPONENTS

Abstract. Indoor point cloud data is easy to obtain and contains a large amount of data, but there is no connectivity, no structural attributes, and no semantic information between point clouds. It is very difficult to extract interior components. In order to solve the problem that it is difficult to extract the closed door in the indoor point cloud, and the indoor scanning point cloud data is noisy and redundant, a point cloud segmentation method is proposed to reduce the amount of data and the impact of noise. Because most of the indoor components are not on the same horizontal plane, different component information can be obtained by slicing the point cloud at different heights. In these slices, the component that always exists is the wall. By comparing other data with the point cloud data with a more complete wall, the required component can be extracted. The experimental results show that this method can accurately extract the indoor door component information and beam component information, providing a data basis for building model refinement.

Establishment of a Coal Mine Roadway Model Based on Point Cloud Feature Matching

A coal mine roadway is a longitudinally limited space with curves and branches, low illumination and high humidity, a large amount of dust, and an unstructured terrain environment. Traditional ICP algorithms have the defects of slow convergence speed and it is easy to fall into local optimums. While the NDT algorithm in the NDT + ICP algorithm has high registration efficiency, poor stability, and low registration accuracy, which are not suitable for point clouds with noise and a large amount of data. By calculating the FPFH value, the detailed description of the point cloud will be greatly increased to increase the robustness and accuracy Therefore, a feature registration method based on the FPFH + ICP algorithm is proposed to reduce the modeling error of excavation roadways and meet the requirements of intelligent excavation. First, outliers caused by dust are treated by the Euclidean clustering point cloud segmentation method, and then the calculation of the normal vector in the FPFH feature descriptor is optimized based on extracting key points from the roadway structure. The surface normal vector of each key point and its neighborhood point is estimated according to the measured point and its neighborhood point. The initial coordinate transformation matrix of a point cloud of an excavated roadway is obtained by the SAC-IA algorithm and transferred to the ICP algorithm. Finally, KD-tree is introduced into the ICP algorithm to accelerate the search speed of corresponding point pairs, and the Gauss–Newton method is used to optimize the solution of the nonlinear objective function of the algorithm to complete accurate registration of point clouds in an excavation roadway.

Multi-phenotypic parameters extraction and biomass estimation for lettuce based on point clouds

Traditional phenotypic measurements were destructive, time-consuming and laborious. 3D computer vision technology collects plant information in a non-destructive manner and can work continuously. The purpose of this paper is to measure lettuce multi-phenotypic parameters and estimate biomass by point clouds. The result shows: (1) RGB information improved the segmentation effect of the point cloud. (2) The point cloud segmentation method proposed in this paper has a high accuracy (accuracy = 98.7 %) and keeps high spatial resolution. (3) On the test set, the R2 of height, diameter, leaf area, fresh weight and dry weight are 0.935, 0.905, 0.969, 0.966 and 0.968, respectively, the cumulative NMSE is as low as 0.069. The Totalscore is 15 % better than the 1st of the online challenge held based on this data set. The results showed that the plant phenotypic extraction method proposed in this paper was accurate and efficient, and could be applied for high-throughput phenotyping.

DEEP LEARNING APPROACHES FOR ANOMALIES DETECTION OF BLADDER CT CONTOURS IN PROSTATE CANCER PATIENTS

In radiation therapy, poor contour delineations of Region of Interests (ROI) may result in incorrect radiation dosing and increase the chance of radiation-induced toxicities. To better recognize these contours, we used a feature fusion-based 3-dimensional convolution neural network (3D-CNN), multi-view convolution neural network (MV-CNN), and 3D point cloud to retrospectively detect abnormal bladder contours among prostate cancer patients. We defined three patterns of abnormal bladder contours: (1) discontinuities: missing slices and disparate volumes, (2) surface aberrations: extended or contracted slices, (3) small volume: bladder volume [Formula: see text][Formula: see text]ml. We created virtual samples for discontinuities and surface aberrations by randomly adding pixel to one or more slices outside the original contours and randomly extending or shrinking one or more slices. For both 3D-CNN and MV-CNN methods, we utilized the Local Binary Pattern (LBP) and Canny operators for feature extraction and compared several feature fusion-based approaches, including Original image + LBP processed image, Original image + Canny processed image, Original image + LBP processed image + Canny processed image. For the 3D point cloud, we applied the 3D point cloud segmentation method (Rand LA-Net) and the 3D point cloud classifying method (DGCNN). We had 344 samples available (219 real contours and 125 virtual contours) altogether. All the algorithms were trained on 200 bladder contours, validated on 100 bladders, then tested on 44 real annotated contours. Among all the approaches, the “Original + LBP”-based MV-CNN model achieved the highest accuracy: 100% for predicting discontinuity and small volume, 91.3% for predicting normal contours, and 90.9% for predicting surface aberrations. MV-CNN performed more accurately and efficiently across all the feature fusion methods. In particular, “Original + LBP”-based MV-CNN achieved the highest test accuracy. Our tools can detect anomalies of bladder contours in clinical settings and may improve the process of contour quality assurance (QA).

Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation

BackgroundMarker-based augmented reality (AR) calibration methods for surgical navigation often require a second computed tomography scan of the patient, and their clinical application is limited due to high manufacturing costs and low accuracy. MethodsThis work introduces a novel type of AR calibration framework that combines a Microsoft HoloLens device with a single camera registration module for surgical navigation. A camera is used to gather multi-view images of a patient for reconstruction in this framework. A shape feature matching-based search method is proposed to adjust the size of the reconstructed model. The double clustering-based 3D point cloud segmentation method and 3D line segment detection method are also proposed to extract the corner points of the image marker. The corner points are the registration data of the image marker. A feature triangulation iteration-based registration method is proposed to quickly and accurately calibrate the pose relationship between the image marker and the patient in the virtual and real space. The patient model after registration is wirelessly transmitted to the HoloLens device to display the AR scene. ResultsThe proposed approach was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with six advanced AR calibration methods. The proposed method obtained average fusion errors of 0.70 ± 0.16 and 0.91 ± 0.13 mm in phantom and volunteer experiments, respectively. The fusion accuracy of the proposed method is the highest among all comparison methods. A volunteer liver puncture clinical simulation experiment was also conducted to show the clinical feasibility. ConclusionsOur experiments proved the effectiveness of the proposed AR calibration method, and revealed a considerable potential for improving surgical performance.