Mesh Segmentation Research Articles

Influence investigation of morphological and distributional properties of surficial aggregates on skid resistance of asphalt pavement

Skid resistance of the asphalt pavement is greatly influenced by the morphology and spatial distribution of surficial aggregates on the pavement. To clarify the correlation between them, this study accurately detects and segments the mesh of aggregates in the pavement model reconstructed by the laser scanner. Subsequently, three morphological indices (the peak height, cross-section form, and root-mean-square gradient of an aggregate) and two distributional indices (the aggregate spacing, and aggregate distribution dispersity) were calculated. Simultaneously, British pendulum number was also measured to represent the skid resistance of the pavement. At last, the correlation between the British pendulum number and five indices of surficial aggregates of the pavement was obtained using the grey relation method. Especially, the mesh segmentation algorithm developed was enhanced in splitting adhesive mesh of adjacent aggregates and restoring incorrectly splitted mesh of aggregates to improve the accuracy and robustness of the algorithm. Six regions on the surface of two rut specimens under SMA-16 and GAC-16 gradations were selected to analyze the correlation between British pendulum number and the five indices. Results indicate that the cross-section form influences the skid resistance of the pavement most, followed by the aggregate spacing, aggregate distribution dispersity, aggregate peak height, and aggregate RMS gradient. This facilitates the potential directions to improve the skid resistance by optimizing the highly relevant indices in future.

Two-Stream Modality-Based Deep Learning Approach for Enhanced Two-Person Human Interaction Recognition in Videos.

Human interaction recognition (HIR) between two people in videos is a critical field in computer vision and pattern recognition, aimed at identifying and understanding human interaction and actions for applications such as healthcare, surveillance, and human-computer interaction. Despite its significance, video-based HIR faces challenges in achieving satisfactory performance due to the complexity of human actions, variations in motion, different viewpoints, and environmental factors. In the study, we proposed a two-stream deep learning-based HIR system to address these challenges and improve the accuracy and reliability of HIR systems. In the process, two streams extract hierarchical features based on the skeleton and RGB information, respectively. In the first stream, we utilised YOLOv8-Pose for human pose extraction, then extracted features with three stacked LSM modules and enhanced them with a dense layer that is considered the final feature of the first stream. In the second stream, we utilised SAM on the input videos, and after filtering the Segment Anything Model (SAM) feature, we employed integrated LSTM and GRU to extract the long-range dependency feature and then enhanced them with a dense layer that was considered the final feature for the second stream module. Here, SAM was utilised for segmented mesh generation, and ImageNet was used for feature extraction from images or meshes, focusing on extracting relevant features from sequential image data. Moreover, we newly created a custom filter function to enhance computational efficiency and eliminate irrelevant keypoints and mesh components from the dataset. We concatenated the two stream features and produced the final feature that fed into the classification module. The extensive experiment with the two benchmark datasets of the proposed model achieved 96.56% and 96.16% accuracy, respectively. The high-performance accuracy of the proposed model proved its superiority.

1D CNNs and face-based random walks: A powerful combination to enhance mesh understanding and 3D semantic segmentation

In this paper, we present a novel face-based random walk method aimed at addressing the 3D semantic segmentation issue. Our method utilizes a one-dimensional convolutional neural network for detailed feature extraction from sequences of triangular faces and employs a stacked gated recurrent unit to gather information along the sequence during training. This approach allows us to effectively handle irregular meshes and utilize the inherent feature extraction potential present in mesh geometry. Our study's results show that the proposed method achieves competitive results compared to the state-of-the-art methods in mesh segmentation. Importantly, it requires fewer training iterations and demonstrates versatility by applying to a wide range of objects without the need for the mesh to adhere to manifold or watertight topology requirements.

A Dataset of Electrical Components for Mesh Segmentation and Computational Geometry Research

Data quality is of crucial importance in the field of automated or digitally assisted assembly. This paper presents a comprehensive data set of triangle meshes representing electrical and electronic components obtained by scraping Computer Aided Design (CAD) models from the Internet. Consisting of a total of 234 triangle meshes with labelled vertices, this data set was specifically created for segmentation tasks. Its versatility for multimodal tasks is underscored by the presence of various labels, including vertex labels, categories, and subcategories. This paper presents the data set and provides a thorough statistical analysis, including measures of shape, size, distribution, and inter-rater reliability. In addition, the paper suggests several approaches for using the data set, considering its multimodal characteristics. The data set and related findings presented in this paper are intended to encourage further research and advancement in the field of manufacturing automation, specifically spatial assembly.

Variational sparse diffusion and its application in mesh processing

PurposeThe primary objective of this study is to tackle the enduring challenge of preserving feature integrity during the manipulation of geometric data in computer graphics. Our work aims to introduce and validate a variational sparse diffusion model that enhances the capability to maintain the definition of sharp features within meshes throughout complex processing tasks such as segmentation and repair.Design/methodology/approachWe developed a variational sparse diffusion model that integrates a high-order L1 regularization framework with Dirichlet boundary constraints, specifically designed to preserve edge definition. This model employs an innovative vertex updating strategy that optimizes the quality of mesh repairs. We leverage the augmented Lagrangian method to address the computational challenges inherent in this approach, enabling effective management of the trade-off between diffusion strength and feature preservation. Our methodology involves a detailed analysis of segmentation and repair processes, focusing on maintaining the acuity of features on triangulated surfaces.FindingsOur findings indicate that the proposed variational sparse diffusion model significantly outperforms traditional smooth diffusion methods in preserving sharp features during mesh processing. The model ensures the delineation of clear boundaries in mesh segmentation and achieves high-fidelity restoration of deteriorated meshes in repair tasks. The innovative vertex updating strategy within the model contributes to enhanced mesh quality post-repair. Empirical evaluations demonstrate that our approach maintains the integrity of original, sharp features more effectively, especially in complex geometries with intricate detail.Originality/valueThe originality of this research lies in the novel application of a high-order L1 regularization framework to the field of mesh processing, a method not conventionally applied in this context. The value of our work is in providing a robust solution to the problem of feature degradation during the mesh manipulation process. Our model’s unique vertex updating strategy and the use of the augmented Lagrangian method for optimization are distinctive contributions that enhance the state-of-the-art in geometry processing. The empirical success of our model in preserving features during mesh segmentation and repair presents an advancement in computer graphics, offering practical benefits to both academic research and industry applications.

Model-based monocular 6-degree-of-freedom pose tracking for asteroid

In this paper, we present a novel vision-based framework to track the 6-DoF pose of an asteroid in real time with the 3D contour of the asteroid as a feature. During pose tracking, at the beginning time of tracking, the tracking system is initialized by a pose retrieval method. At each subsequent time instant, given the 3D mesh model of an asteroid, with the initial pose and its covariance given by the square root cubature Kalman Filter (SCKF), the 3D mesh segments constituting the 3D asteroid contour are efficiently extracted from the 3D mesh model. Then, in the input asteroid image, we search the image points corresponding to the extracted 3D segments within the searching range defined by the initial pose and its covariance. After that, the asteroid pose is determined in real time by minimizing the angles between the back-projection lines of the searched image points and the projection planes of the corresponding 3D segments, which is much more robust to the position change of the asteroid and asteroid size. The covariance matrix of the pose is inferred from the Cartesian noise model in the first order. Eventually, the SCKF is derived from the second-order auto regression to generate the final pose estimate and give the initial pose and its covariance for the next time instant. The synthetic trials quantitatively validate the real-time performance, robustness, and accuracy of our algorithm in dark space, different imaging distances, lighting conditions, image noise, model error, and initial pose error, and meanwhile, the real trial qualitatively shows the effectiveness of our method.

A prospective, multicentre, registry study of RECO in the endovascular treatment of acute ischaemic stroke

The RECO is a novel endovascular treatment (EVT) device that adjusts the distance between two mesh segments to axially hold the thrombus. We organized this postmarket study to assess the safety and performance of RECO in acute ischaemic stroke (AIS) patients with large vessel occlusion (LVO). This was a single-arm prospective multicentre study that enrolled patients as first-line patients treated with RECO at 9 stroke centres. The primary outcome measures included functional independence at 90 days (mRS 0–2), symptomatic intracranial haemorrhage (sICH), time from puncture to recanalization and time from symptom onset to recanalization. The secondary outcome measures were a modified thrombolysis in cerebral infarction (mTICI) score of 2b or 3 after the first attempt and at the end of the procedure and the all-cause mortality rate within 90 days. From May 22, 2020, to July 30, 2022, a total of 268 consecutive patients were enrolled in the registry. The median puncture-to-recanalization time was 64 (IQR, 45–92), and the symptom onset-to-recanalization time was 328 min (IQR, 228–469). RECO achieved successful reperfusion (mTICI 2b-3) after the first pass in 133 of 268 patients (49.6%). At the end of the operation, 96.6% of the patients reached mTICI 2b-3, and 97.4% of the patients ultimately achieved successful reperfusion. Sixteen (7.2%) patients had sICH. A total of 132 (49.3%) patients achieved functional independence at 90 days, and the all-cause mortality rate within 90 days was 17.5%. In this clinical experience, the RECO device achieved a high rate of complete recanalization with a good safety profile and favourable 90-day clinical outcomes.Clinical trial registration: URL: https://www.clinicaltrials.gov/; Unique identifier: NCT04840719.

Mesh Segmentation for Individual Teeth Based on Two-Stream GCN With Self-Attention

Mesh Segmentation for Individual Teeth Based on Two-Stream GCN With Self-Attention

MeshNet-SP: A Semantic Urban 3D Mesh Segmentation Network with Sparse Prior

A textured urban 3D mesh is an important part of 3D real scene technology. Semantically segmenting an urban 3D mesh is a key task in the photogrammetry and remote sensing field. However, due to the irregular structure of a 3D mesh and redundant texture information, it is a challenging issue to obtain high and robust semantic segmentation results for an urban 3D mesh. To address this issue, we propose a semantic urban 3D mesh segmentation network (MeshNet) with sparse prior (SP), named MeshNet-SP. MeshNet-SP consists of a differentiable sparse coding (DSC) subnetwork and a semantic feature extraction (SFE) subnetwork. The DSC subnetwork learns low-intrinsic-dimensional features from raw texture information, which increases the effectiveness and robustness of semantic urban 3D mesh segmentation. The SFE subnetwork produces high-level semantic features from the combination of features containing the geometric features of a mesh and the low-intrinsic-dimensional features of texture information. The proposed method is evaluated on the SUM dataset. The results of ablation experiments demonstrate that the low-intrinsic-dimensional feature is the key to achieving high and robust semantic segmentation results. The comparison results show that the proposed method can achieve competitive accuracies, and the maximum increase can reach 34.5%, 35.4%, and 31.8% in mR, mF1, and mIoU, respectively.

Automatic Detection of Tooth-Gingiva Trim Lines on Dental Surfaces.

Detecting the tooth-gingiva trim line from a dental surface plays a critical role in dental treatment planning and aligner 3D printing. Existing methods treat this task as a segmentation problem, which is resolved with geometric deep learning based mesh segmentation techniques. However, these methods can only provide indirect results (i.e., segmented teeth) and suffer from unsatisfactory accuracy due to the incapability of making full use of high-resolution dental surfaces. To this end, we propose a two-stage geometric deep learning framework for automatically detecting tooth-gingiva trim lines from dental surfaces. Our framework consists of a trim line proposal network (TLP-Net) for predicting an initial trim line from the low-resolution dental surface as well as a trim line refinement network (TLR-Net) for refining the initial trim line with the information from the high-resolution dental surface. Specifically, our TLP-Net predicts the initial trim line by fusing the multi-scale features from a U-Net with a proposed residual multi-scale attention fusion module. Moreover, we propose feature bridge modules and a trim line loss to further improve the accuracy. The resulting trim line is then fed to our TLR-Net, which is a deep-based LDDMM model with the high-resolution dental surface as input. In addition, dense connections are incorporated into TLR-Net for improved performance. Our framework provides an automatic solution to trim line detection by making full use of raw high-resolution dental surfaces. Extensive experiments on a clinical dental surface dataset demonstrate that our TLP-Net and TLR-Net are superior trim line detection methods and outperforms cutting-edge methods in both qualitative and quantitative evaluations.

A Critical Analysis of the Limitation of Deep Learning based 3D Dental Mesh Segmentation Methods in Segmenting Partial Scans.

Tooth segmentation from intraoral scans is a crucial part of digital dentistry. Many Deep Learning based tooth segmentation algorithms have been developed for this task. In most of the cases, high accuracy has been achieved, although, most of the available tooth segmentation techniques make an implicit restrictive assumption of full jaw model and they report accuracy based on full jaw models. Medically, however, in certain cases, full jaw tooth scan is not required or may not be available. Given this practical issue, it is important to understand the robustness of currently available widely used Deep Learning based tooth segmentation techniques. For this purpose, we applied available segmentation techniques on partial intraoral scans and we discovered that the available deep Learning techniques under-perform drastically. The analysis and comparison presented in this work would help us in understanding the severity of the problem and allow us to develop robust tooth segmentation technique without strong assumption of full jaw model.Clinical relevance- Deep learning based tooth mesh segmentation algorithms have achieved high accuracy. In the clinical setting, robustness of deep learning based methods is of utmost importance. We discovered that the high performing tooth segmentation methods under-perform when segmenting partial intraoral scans. In our current work, we conduct extensive experiments to show the extent of this problem. We also discuss why adding partial scans to the training data of the tooth segmentation models is non-trivial. An in-depth understanding of this problem can help in developing robust tooth segmentation tenichniques.

MWFormer: Mesh Understanding with Window-based Transformer

Polygonal mesh has been proven to be a powerful representation of 3D shapes, given its efficiency in expressing shape surface while maintaining geometric and topological information. Increasing efforts have been made to design elaborate deep convolutional neural networks for meshes. However, these methods naturally ignore the global connectivity among mesh primitives due to the locality nature of convolution operations. In this paper, we introduce a transformer-like self-attention mechanism with down-sampling architectures for mesh learning to capture both the global and local relationships among mesh faces. To achieve this, we propose BFS-Pooling, which can convert a connected mesh into discrete tokens (i.e., a set of adjacent faces) with breath-first-search (BFS) and naturally build hierarchical architectures for mesh learning by pooling mesh tokens. Benefiting from BFS-Pooling, we design a hierarchical transformer architecture with a window-based local attention mechanism, Mesh Window Transformer (MWFormer). Experimental results demonstrate that MWFormer achieves the best or competitive performance in both mesh classification and mesh segmentation tasks. Code will be available.

MeT: A graph transformer for semantic segmentation of 3D meshes

Polygonal meshes have become the standard for discretely approximating 3D shapes, thanks to their efficiency and high flexibility in capturing non-uniform shapes. This non-uniformity, however, leads to irregularity in the mesh structure, making tasks like segmentation of 3D meshes particularly challenging. Semantic segmentation of 3D mesh has been typically addressed through CNN-based approaches, leading to good accuracy. Recently, transformers have gained enough momentum both in NLP and computer vision fields, achieving performance at least on par with CNN models, supporting the long-sought architecture universalism. Following this trend, we propose a transformer-based method for semantic segmentation of 3D mesh motivated by a better modeling of the graph structure of meshes, by means of global attention mechanisms. In order to address the limitations of standard transformer architectures in modeling relative positions of non-sequential data, as in the case of 3D meshes, as well as in capturing the local context, we perform positional encoding by means the Laplacian eigenvectors of the adjacency matrix, replacing the traditional sinusoidal positional encodings, and by introducing clustering-based features into the self-attention and cross-attention operators. Experimental results, carried out on three sets of the Shape COSEG Dataset (Wang et al., 2012), on the human segmentation dataset proposed in Maron et al. (2017) and on the ShapeNet benchmark (Chang et al., 2015), show how the proposed approach yields state-of-the-art performance on semantic segmentation of 3D meshes.

Deep learning-based semantic segmentation of urban-scale 3D meshes in remote sensing: A survey

Semantic segmentation in 3D meshes is the classification of its constituent element(s) into specific classes or categories. Using the powerful feature extraction abilities of deep neural networks (DNNs), significant results have been obtained in the semantic segmentation of various remotely sensed data formats. With the increased utilization of DNNs to segment remotely sensed data, there have been commensurate in-depth reviews and surveys summarizing the various learning-based techniques and methodologies that entail these methods. However, most of these surveys focused on methods that involve popular data formats like LiDAR point clouds, synthetic aperture radar (SAR) images, and hyperspectral images (HSI) while 3D meshes hardly received any attention. In this paper, to our best knowledge, we present the first comprehensive and contemporary survey of recent advances in utilizing deep learning techniques for the semantic segmentation of urban-scale 3D meshes. We first describe the different approaches employed by mesh-based learning methods to generalize and implement learning techniques on the mesh surface, and then describe how the element-wise classification tasks are achieved through these methods. We also provide an in-depth discussion and comparative analysis of the surveyed methods followed by a summary of the benchmark large-scale mesh datasets accompanied with the evaluation metrics for assessing the segmentation performance of the methods. Finally, we summarize some of the contemporary problems of the field and provide future research directions that may help researchers in the community.

Unsupervised Contrastive Representation Learning for 3D Mesh Segmentation (Student Abstract)

3D deep learning is a growing field of interest due to the vast amount of information stored in 3D formats. Triangular meshes are an efficient representation for irregular, non-uniform 3D objects. However, meshes are often challenging to annotate due to their high computational complexity. Therefore, it is desirable to train segmentation networks with limited-labeled data. Self-supervised learning (SSL), a form of unsupervised representation learning, is a growing alternative to fully-supervised learning which can decrease the burden of supervision for training. Specifically, contrastive learning (CL), a form of SSL, has recently been explored to solve limited-labeled data tasks. We propose SSL-MeshCNN, a CL method for pre-training CNNs for mesh segmentation. We take inspiration from prior CL frameworks to design a novel CL algorithm specialized for meshes. Our preliminary experiments show promising results in reducing the heavy labeled data requirement needed for mesh segmentation by at least 33%.

Design and Implementation of Evaluation Method for Spraying Coverage Region of Plant Protection UAV

Plant protection UAVs are becoming the preferred plant protection method for agricultural pest control. At present, the evaluation of droplet distribution in aerial spraying is collected and evaluated after the completion of prevention and control operations, and there is a lack of real-time evaluation methods. Based on the flight parameter during the UAV plant protection process, real-time estimation of droplet distribution is the key to solving this problem and further improving the effectiveness of aerial spraying. This study proposes a merging algorithm for arbitrary polygonal regions, meshing the boundaries of the region, divide the mesh segments based on the overlapping meshes between the two regions, and connect the valid mesh connection segments of the two regions according to certain rules to obtain the intersection, union, and residual operation results between the regions. Afterwards, software based on this algorithm was developed and applied to generate spraying coverage regions, leakage spray regions, and repeated spray regions. The experimental results on theoretical and irregular routes show that the algorithm can accurately generate droplet distribution regions. The error of the calculation results with a mesh scale of 0.05 m is within 7‰, and the operating speed is above 30 Hz, meeting the real-time requirements. The smaller the mesh scale is, the higher the accuracy of the calculation results is, but the slower the calculation speed. Therefore, in practical applications, it is necessary to choose an appropriate mesh scale based on hardware computing power and accuracy level requirements. This study solves the problem of cumulative calculation of droplet distribution during the operation of plant protection UAVs, providing a basis for objectively evaluating the operation quality of plant protection UAVs and optimizing the setting of operation parameters.

A data-centric unsupervised 3D mesh segmentation method

A data-centric unsupervised 3D mesh segmentation method

Part-to-Surface Mesh Segmentation for Mechanical Models Based on Multi-Stage Clustering

In this paper, we propose a part-to-surface mesh segmentation method for engineering models characterized by a multi-stage clustering of surface patches generated by the Variational Shape Approximation (VSA) method. Initially, border types between topologically adjacent surface patches are estimated and surface patches bonded by certain border types are grouped into clusters. Secondly, clusters of surface patches are iteratively aggregated into volumetric part-level sub-meshes according to the convexity variations of intermediate sub-meshes. At last, surface patches within their respective part-level sub-meshes are merged into surface-level sub-meshes that are well-fitted by basic primitive shapes based on an improved surface-aware similarity metric. Experimental results demonstrate that the proposed method can achieve better results on mechanical models both quantitatively and perceptually, especially for objects with non-zero genus topology and rich in tiny features.

SCMS-Net: Self-Supervised Clustering-Based 3D Meshes Segmentation Network

The superior performance of deep learning in different domains has sparked significant interest in its applicability to 3D computer graphics. Deep learning has become the dominant technical architecture in current 3D mesh segmentation. However, learning-based 3D segmentation methods usually rely on high-quality training datasets, which are not readily available in practical applications. How to segment 3D meshes without exhaustive label annotations remains a challenging problem, especially in the context of deep learning. As a subset of unsupervised learning methods, self-supervised learning offers a promising learning paradigm for unlabeled 3D mesh segmentation. In this paper, we introduce a self-supervised clustering-Based network specifically for the segmentation of label-free 3D meshes. Our self-supervised clustering-based 3D mesh segmentation network (SCMS-Net) employs a two-branch architecture to learn effective feature representation. The two branches are unified into an end-to-end framework using a self-supervised strategy. Finally, the label predictions of the parts are generated by iterative clustering. We conducted ablation studies and comparative experiments on a standard benchmark to demonstrate the effectiveness of our approach.

Mesh Convolutional Networks With Face and Vertex Feature Operators.

Deep learning techniques have proven effective in many applications, but these implementations mostly apply to data in one or two dimensions. Handling 3D data is more challenging due to its irregularity and complexity, and there is a growing interest in adapting deep learning techniques to the 3D domain. A recent successful approach called MeshCNN consists of a set of convolutional and pooling operators applied to the edges of triangular meshes. While this approach produced superb results in classification and segmentation of 3D shapes, it can only be applied to edges of a mesh, which can constitute a disadvantage for applications where the focuses are other primitives of the mesh. In this study, we propose face-based and vertex-based operators for mesh convolutional networks. We design two novel architectures based on the MeshCNN network that can operate on faces and vertices of a mesh, respectively. We demonstrate that the proposed face-based architecture outperforms the original MeshCNN implementation in mesh classification and mesh segmentation, setting the new state of the art on benchmark datasets. In addition, we extend the vertex-based operator to fit in the Point2Mesh model for mesh reconstruction from clean, noisy, and incomplete point clouds. While no statistically significant performance improvements are observed, the model training and inference time are reduced by the proposed approach by 91% and 20%, respectively, as compared with the original Point2Mesh model.