3D Mesh Model Research Articles

Interactive segmentation with curve-based template deformation for spatiotemporal computed tomography of swallowing motion.

Repeating X-ray computed tomography (CT) measurements over a short period of time allows for obtaining a spatiotemporal four-dimensional (4D) volume image. This study presents an interactive method for segmenting a 4DCT image by fitting a template model to a target organ. The template consists of a three-dimensional (3D) mesh model and free-form-deformation (FFD) cage enclosing the mesh. The user deforms the template by placing multiple curve constraints that specify the boundary shape of the template in 3D space. We also present curve constraints shared over all time frames and interpolated along the time axis to facilitate efficient curve specification. Our method formulates the template deformation using the FFD cage modification, allowing the user to switch between our curve-based method and traditional FFD at any time. To illustrate the feasibility of our method, we show segmentation results in which we could accurately segment three organs from a 4DCT image capturing a swallowing motion. To evaluate the usability of our method, we conducted a user study comparing our curve-based method with the cage-based FFD. We found that the participants finished segmentation in approximately 20% interaction time periods on average with our method.

Generating a virtual acoustic environment to approximate the soundscape of a historical cathedral

This paper focuses on the generation of a virtual acoustic environment of a historical cathedral basilica located in St. Augustine, Florida, USA. The virtual acoustic environment is an effective means to study a cultural heritage's soundscape that restricts visitors from accessing and being quiet. The author used a laser scanner to capture the reality of the cathedral's interior, in order to maximize the visual perception of the worship space, and created a 3D digital textured mesh model using the points cloud generated from laser scanning. The spatial experience recorded by a 360 camera with an ambisonic microphone provided the virtual soundwalk at multiple locations. The background and foreground sound were synthesized in Unity using the same music recordings that the church used during the day. The sound pressure levels were closely matched with the ones measured by a sound level meter. The virtual acoustic environment was simulated by using spatial applications in Unity and optimized through the comparison of the room acoustical measurements performed based on ISO 3382.

3D textured mesh scene data fusion method considering multiple scales and resolutions

ABSTRACT The fusion of 3D textured mesh scene data can effectively and seamlessly integrate multiple spatially related 3D geographical scene models into larger and more comprehensive datasets. Existing methods usually have different degrees of texture distortion and unreduced texture problems in the splicing area, and these methods do not consider the current situation of Level of Detail(LoD) data organisation in the 3D textured mesh model. We propose a 3D textured mesh scene data fusion method considering multiple scales and resolutions. First, seamless extraction of splicing areas was achieved based on spatial relationship constraints and orthogonal inverse projection principles. Second, the sparse scene structure concept was employed to generate sparse summary scene images. Finally, a dual simplification operator for both mesh and texture was proposed to construct multi-resolution 3D textured mesh models, simultaneously reconstructing triangular facets and textures within the splicing regions. We used multiple 3D textured mesh data constructed from aerial obliques to conduct data fusion experimental verification. The results indicate that our method exhibits minimal texture distortion during grid reconstruction in splicing areas and model simplification, effectively reducing the amount of simplified texture data. Additionally, our method adeptly addresses multilevel detailed 3D textured mesh models, demonstrating good feasibility and advancement.

Improvement of ankle foot orthotics fabrication using 3D printing method

Orthotics are the body support devices used for correction, immobilization, fixation, and prevention of paralysis. The greatest number of orthotics utilized by people suffering from plantarflexion and dorsiflexion disability, especially in Indonesia, is ankle foot orthotic (AFO). However, the duration associated with fabricating AFO through conventional methods is considered time-consuming. This paper aims to fabricate ankle foot orthotics (AFO) using innovative combinations of 3D scanning and 3D printing. The method begins with 3D scanning of the patient’s lower limb using photogrammetry (3DF Zephyr). The design is generated and adjusted, afterwards, the orthotic prototype is produced by fused deposition modeling (FDM) 3D printing using polypropylene (PP) material. This choice is attributed to the material's advantages, such as being lightweight, rigid, durable, and cost-effective. The 3D mesh model scanned using 3DF Zephyr shows good quality and more precise results. In addition, the prototype produced using 3D printing was tested by walking based on normal gait analysis’s angle of foot and calf measurement, which shows a maximum range of motion (ROM) of 16.1˚. The proposed methods of fabricating orthotic prototypes can successfully reduce the processing time by approximately 70% compared to the conventional method.

Accurately Computing the Interacted Volume of Molecules over Their 3D Mesh Models.

For quickly predicting the rational arrangement of catalysts and substrates, we previously proposed a method to calculate the interacted volumes of molecules over their 3D point cloud models. However, the nonuniform density in molecular point clouds may lead to incomplete contours in some slices, reducing the accuracy of the previous method. In this paper, we propose a two-step method for more accurately computing molecular interacted volumes. First, by employing a prematched mesh slicing method, we layer the 3D triangular mesh models of the electrostatic potential isosurfaces of two molecules globally, transforming the volume calculation into finding the intersecting areas in each layer. Next, by subdividing polygonal edges, we accurately identify intersecting parts within each layer, ensuring precise calculation of interacted volumes. In addition, we present a concise overview for computing intersecting areas in cases of multiple contour intersections and for improving computational efficiency by incorporating bounding boxes at three stages. Experimental results demonstrate that our method maintains high accuracy in different experimental data sets, with an average relative error of 0.16%. On the same experimental setup, our average relative error is 0.07%, which is lower than the previous algorithm's 1.73%, improving the accuracy and stability in calculating interacted volumes.

Reversible Data Hiding in Encrypted 3D Mesh Models Based on Multi-Group Partition and Closest Pair Prediction

The reversible data hiding scheme in the encrypted domain is a potential solution to the concerns regarding user privacy in cloud applications. The 3D mesh model is an emerging file format and is widely used in engineering modeling, special effects, and video games. However, studies on reversible data hiding in encrypted 3D mesh models are still in the preliminary stage. In this paper, two novel techniques, multi-group partition (MGP) and closest pair prediction (CPP), are proposed to improve performance. The MGP technique adaptively classifies vertices into reference and embeddable vertices, while the CPP technique efficiently predicts embeddable vertices and generates shorter recovery information to vacate more redundancy for additional data embedding. Experimental results indicate that the proposed scheme significantly improves the embedding rate compared to state-of-the-art schemes and can be used in real-time applications.

Design and Validation of a Novel 3D-Printed Retrograde Intrarenal Surgery Trainer

ObjectivesTo report on the design of a novel 3D-printed retrograde intrarenal surgery (RIRS) benchtop trainer and detail its validation against real-life experiences. MethodsDigital Imaging and Communications in Medicine (DICOM) files of two patients with normal computed tomography of the kidney and bladder were converted into stereolithography files to create 3D triangular mesh models. These images were further refined using Autodesk Meshmixer. These 3D models were fabricated through additive manufacturing, a process commonly known as 3D printing and assembled in a polypropylene case. After development, the model was validated by 40 experienced urologists and urology residents in their final year of training. They were asked to rate the components of the simulation using a nine-point questionnaire. ResultsThe model’s value in understanding the principles of RIRS and simulating contextual anatomy had mean scores of 9.43 (standard deviation [SD] = 0.74) and 9.21 (SD = 1.03), respectively. Mean scores for specific steps in RIRS were 8.07 (SD 1.47) for cannulating the ureteric orifice, 8.61 (SD 1.24) for inserting the ureteric access sheath, 9.29 (SD 0.97) for performing a renoscopy and evaluating all the calyces, 9.46 (SD 0.87) for laser lithotripsy, and 9.17 (SD 0.94) for manual stone retrieval. Participants scored the model with a mean score of 9.04 (SD 0.87) regarding realism and a mean score of 9.18 (SD 0.89) when evaluating its ability to enhance a trainee’s confidence in RIRS. ConclusionsThe model performed well for all components of RIRS. This model allows high fidelity of the simulation, and is cost-effective, portable, durable, reusable, and compatible with standard ureteroscopes.

MOLO-SLAM: A Semantic SLAM for Accurate Removal of Dynamic Objects in Agricultural Environments

Visual simultaneous localization and mapping (VSLAM) is a foundational technology that enables robots to achieve fully autonomous locomotion, exploration, inspection, and more within complex environments. Its applicability also extends significantly to agricultural settings. While numerous impressive VSLAM systems have emerged, a majority of them rely on static world assumptions. This reliance constrains their use in real dynamic scenarios and leads to increased instability when applied to agricultural contexts. To address the problem of detecting and eliminating slow dynamic objects in outdoor forest and tea garden agricultural scenarios, this paper presents a dynamic VSLAM innovation called MOLO-SLAM (mask ORB label optimization SLAM). MOLO-SLAM merges the ORBSLAM2 framework with the Mask-RCNN instance segmentation network, utilizing masks and bounding boxes to enhance the accuracy and cleanliness of 3D point clouds. Additionally, we used the BundleFusion reconstruction algorithm for 3D mesh model reconstruction. By comparing our algorithm with various dynamic VSLAM algorithms on the TUM and KITTI datasets, the results demonstrate significant improvements, with enhancements of up to 97.72%, 98.51%, and 28.07% relative to the original ORBSLAM2 on the three datasets. This showcases the outstanding advantages of our algorithm.

Spatial virtual recovery design technology of ancient buildings based on 3D laser scanning technology

Due to the persistent advancements in science and technology, the implementation of digital technology in the preservation of cultural heritage is progressively expanding. One of the focal points of research lies in the spatial virtual restoration design technology of ancient edifices, which is based on three-dimensional laser scanning technology. The study initially outlines the process of obtaining point cloud data via 3D laser scanning technology and subsequently executes denoising, splicing, and simplification processing on the point cloud. Subsequently, the pre-processed data undergo 3D mesh model construction. Finally, an effective repair technique is implemented to address the void phenomenon present during the modeling process. To construct the implicit surface, the RBF method is employed and new vertices are adjusted, resulting in a more accurate spatial virtual restoration of ancient buildings (ABs). The study found that using the radial basis function based void repair method resulted in a mean void repair accuracy of 92.38% in the west wall, an improvement of 3.08% compared to the Liepa-based method. In addition, this method achieved the highest accuracy of 98.94% on the north wall, improving by 6.37% compared to the Poisson grid editing algorithm. Meanwhile, the RBF-based method repaired cavities in the west wall model with an average runtime of only 20.613 s, resulting in a 19.16 s reduction compared to the Liepa-based method. In addition, the method’s average repair time for the north wall was only 5.364 s, a decrease of 13.28 s compared to the Poisson grid editing algorithm. This shows that combining 3D laser scanning technology and cavity repair technology can acquire high-quality point cloud data of historic structures, create precise 3D models, and achieve spatial virtual restoration. This offers a new and efficient technological approach to preserving and passing down ABs.

3D scanning and modeling of highly detailed and geometrically complex historical architectural objects: the example of the Juma Mosque in Khiva (Uzbekistan)

The city centre of Khiva (Uzbekistan), called Itchan Kala, is an architectural complex included in the UNESCO list of tangible cultural heritage. One of the historic buildings in it is the Juma Mosque. It has a simple rectangular structure, but is very large and has 213 deeply carved wooden columns supporting the roof. The article presents the process, problems, and their solutions resulting from the implementation of 3D laser scanning of such highly detailed and geometrically complex historical architectural objects in the conditions of normal tourist traffic. The optimisation of scanning positions, scanning in situ implementation, as well as the processing of the acquired data and the construction of a 3D mesh model of the mosque interior are presented. It is pointed out that scanning such objects with high accuracy and density of measurement points causes major technical problems related to the workload, and the huge volume of data acquired and processed. The possibilities of making the 3D model available in digital space for the purpose of researching the appearance and geometry of the mosque, its individual columns, as well as popularising the monument are also discussed. It is highly probable that the scanning of the Juma Mosque's interior presented here was carried out for the first time in history.

Levenberg–Marquardt deep neural watermarking for 3D mesh using nearest centroid salient point learning

Watermarking is one of the crucial techniques in the domain of information security, preventing the exploitation of 3D Mesh models in the era of Internet. In 3D Mesh watermark embedding, moderately perturbing the vertices is commonly required to retain them in certain pre-arranged relationship with their neighboring vertices. This paper proposes a novel watermarking authentication method, called Nearest Centroid Discrete Gaussian and Levenberg–Marquardt (NCDG–LV), for distortion detection and recovery using salient point detection. In this method, the salient points are selected using the Nearest Centroid and Discrete Gaussian Geometric (NC–DGG) salient point detection model. Map segmentation is applied to the 3D Mesh model to segment into distinct sub regions according to the selected salient points. Finally, the watermark is embedded by employing the Multi-function Barycenter into each spatially selected and segmented region. In the extraction process, the embedded 3D Mesh image is extracted from each re-segmented region by means of Levenberg–Marquardt Deep Neural Network Watermark Extraction. In the authentication stage, watermark bits are extracted by analyzing the geometry via Levenberg–Marquardt back-propagation. Based on a performance evaluation, the proposed method exhibits high imperceptibility and tolerance against attacks, such as smoothing, cropping, translation, and rotation. The experimental results further demonstrate that the proposed method is superior in terms of salient point detection time, distortion rate, true positive rate, peak signal to noise ratio, bit error rate, and root mean square error compared to the state-of-the-art methods.

Development of Automatic Assessment Framework for Spine Deformity Using Freehand 3-D Ultrasound Imaging System.

A 3-D ultrasound (US) imaging technique has been studied to facilitate the diagnosis of spinal deformity without radiation. The objective of this article is to propose an assessment framework to automatically estimate spinal deformity in US spine images. The proposed framework comprises four major components, a US spine image generator, a novel transformer-based lightweight spine detector network, an angle evaluator, and a 3-D modeler. The principal component analysis (PCA) and discriminative scale space tracking (DSST) method are first adopted to generate the US spine images. The proposed detector is equipped with a redundancy queries removal (RQR) module and a regularization item to realize accurate and unique detection of spine images. Two clinical datasets, a total of 273 images from adolescents with idiopathic scoliosis, are used for the investigation of the proposed framework. The curvature is estimated by the angle evaluator, and the 3-D mesh model is established by the parametric modeling technique. The accuracy rate (AR) of the proposed detector can be achieved at 99.5%, with a minimal redundancy rate (RR) of 1.5%. The correlations between automatic curve measurements on US spine images from two datasets and manual measurements on radiographs are 0.91 and 0.88, respectively. The mean absolute difference (MAD) and standard deviation (SD) are 2.72° ± 2.14° and 2.91° ± 2.36° , respectively. The results demonstrate the effectiveness of the proposed framework to advance the application of the 3-D US imaging technique in clinical practice for scoliosis mass screening and monitoring.

3D Reconstruction of Wheat Plants by Integrating Point Cloud Data and Virtual Design Optimization

The morphology and structure of wheat plants are intricate, containing numerous tillers, rich details, and significant cross-obscuration. Methods of effectively reconstructing three-dimensional (3D) models of wheat plants that reflects the varietal architectural differences using measured data is challenging in plant phenomics and functional–structural plant models. This paper proposes a 3D reconstruction technique for wheat plants that integrates point cloud data and virtual design optimization. The approach extracted single stem number, growth position, length, and inclination angle from the point cloud data of a wheat plant. It then built an initial 3D mesh model of the plant by integrating a wheat 3D phytomer template database with variety resolution. Diverse 3D wheat plant models were subsequently virtually designed by iteratively modifying the leaf azimuth, based on the initial model. Using the 3D point cloud of the plant as the overall constraint and setting the minimum Chamfer distance between the point cloud and the mesh model as the optimization objective, we obtained the optimal 3D model as the reconstruction result of the plant through continuous iterative calculation. The method was validated using 27 winter wheat plants, with nine varieties and three replicates each. The R2 values between the measured data and the reconstructed plants were 0.80, 0.73, 0.90, and 0.69 for plant height, crown width, plant leaf area, and coverage, respectively. Additionally, the Normalized Root Mean Squared Errors (NRMSEs) were 0.10, 0.12, 0.08, and 0.17, respectively. The Mean Absolute Percentage Errors (MAPEs) used to investigate the vertical spatial distribution between the reconstructed 3D models and the point clouds of the plants ranged from 4.95% to 17.90%. These results demonstrate that the reconstructed 3D model exhibits satisfactory consistency with the measured data, including plant phenotype and vertical spatial distribution, and accurately reflects the characteristics of plant architecture and spatial distribution for the utilized wheat cultivars. This method provides technical support for research on wheat plant phenotyping and functional–structural analysis.

Model-based monocular 6-degree-of-freedom pose tracking for asteroid

In this paper, we present a novel vision-based framework to track the 6-DoF pose of an asteroid in real time with the 3D contour of the asteroid as a feature. During pose tracking, at the beginning time of tracking, the tracking system is initialized by a pose retrieval method. At each subsequent time instant, given the 3D mesh model of an asteroid, with the initial pose and its covariance given by the square root cubature Kalman Filter (SCKF), the 3D mesh segments constituting the 3D asteroid contour are efficiently extracted from the 3D mesh model. Then, in the input asteroid image, we search the image points corresponding to the extracted 3D segments within the searching range defined by the initial pose and its covariance. After that, the asteroid pose is determined in real time by minimizing the angles between the back-projection lines of the searched image points and the projection planes of the corresponding 3D segments, which is much more robust to the position change of the asteroid and asteroid size. The covariance matrix of the pose is inferred from the Cartesian noise model in the first order. Eventually, the SCKF is derived from the second-order auto regression to generate the final pose estimate and give the initial pose and its covariance for the next time instant. The synthetic trials quantitatively validate the real-time performance, robustness, and accuracy of our algorithm in dark space, different imaging distances, lighting conditions, image noise, model error, and initial pose error, and meanwhile, the real trial qualitatively shows the effectiveness of our method.

ACCURACY RESEARCH OF THE EFFECT OF LOCAL SURFACE MODELING METHODS ON THE GENERATION OF 3D MODELS OF HISTORICAL OBJECTS WITH ROCK-BASED BUILDING STRUCTURES USING OPEN-SOURCE SOFTWARE

Abstract. The paper describes the effect of the local surface modelling methods known as plane, quadric and triangulation on the generation of the 3D mesh models of the historical objects with rock-based building structures using Cloudcompare, an open-source software for point cloud data processing. The methodology begins with data collection of the 3D test objects using terrestrial laser scanning technology, the pre-processing of the point cloud data, the point cloud subsampling process, the process of generating of the local surface modelling data, the process of generating the 3D mesh models and finally, the analysis, which involves the 3D surface deviation analysis between the generated 3D mesh models. The overall results shows that there are no significant differences between the 3D mesh models that was generated from all the three local surface modelling methods. The histogram analysis shows that the plane and the quadric local surface modelling methods is the best methods to be used in the research where the 3D test object to be modelled contains curvy surfaces without sharp edges and corners.

EVALUATION OF NERF 3D RECONSTRUCTION FOR ROCK ART DOCUMENTATION

Abstract. Digital documentation of rock art traditionally relies on a point cloud captured by a terrestrial laser scanner (TLS) or derived from an oriented image obtained using photogrammetry. In modern photogrammetry, the dense point cloud is generated using multi-view stereo (MVS) and subsequently used to generate a photorealistic 3D model. A recent method to reconstruct 3D models from images is Neural Radiance Fields (NeRF), which uses volume density to render the scenes through neural networks. The advantage of NeRF is that it can construct 3D models faster without using high computer processors and memory. NeRF has been studied in various applications, including cultural heritage, but not specifically for rock art documentation. Therefore, this paper evaluates three-dimensional (3D) reconstruction techniques using NeRF on Nerfstudio platform on two rock art datasets and compares them with the point cloud and 3D mesh models obtained from TLS and photogrammetry/MVS. The results have shown that NeRF does not match MVS in achieving geometric precision and texture quality. However, its learning-based approach accelerates reconstruction and offers potential enhancements to complement photogrammetric workflow.

Three-dimensional analysis of mandibular morphology asymmetry and temporomandibular joint position in patients with unilateral Brodie bite.

Previous studies have shown unilateral posterior crossbite is associated with mandibular asymmetry in morphology and position. However, it remains unclear whether unilateral Brodie bite plays a similar role in mandibular development. Therefore, this study aims to investigate the morphological and positional symmetry of mandibles in patients with unilateral Brodie bite by three-dimensional anaylsis. Fourteen patients with unilateral Brodie bite (mean age 18.43 ± 4.24 years) and fourteen sex- and age-matched patients with normal occlusion (mean age 18.07 ± 5.48 years) underwent cone-beam computed tomography (CBCT) scans. 3D surface mesh models of their mandibles were established using Mimics Research 19.0. The surface matching percentage was compared between the original and mirrored mandible by Geomagic Control X software. Furthermore, the dimension and position of the temporomandibular joint (TMJ) were determined for both groups using InVivoDental 5.0. For surface-to-surface deviation analysis, the percentage of mismatch in patients with unilateral Brodie bite was significantly higher than the control group at ±0.50 mm, ±0.75 mm, and ±1.00 mm tolerance (P < .001). In patients with unilateral Brodie syndrome, the condyles on the scissors-bite side showed a significantly more anterior position (P = .03), greater medial inclination (P < .01), and larger posterior TMJ space (P = .01) than the non-scissors-bite side. Patients with unilateral Brodie bite exhibit a more asymmetrical mandibular morphology, with a greater anterior condylar position and posterior joint space on the scissors-bite side, indicating that early diagnosis and treatment may be necessary for patients with unilateral Brodie bite.

Reversible Data Hiding for Encrypted 3D Mesh Models With Secret Sharing Over Galois Field

Reversible Data Hiding for Encrypted 3D Mesh Models With Secret Sharing Over Galois Field

Practical Applications of a Set-Based Camera Deployment Methodology.

This work establishes a complete methodology for solving continuous sets of camera deployment solutions for automated machine vision inspection systems in industrial manufacturing facilities. The methods presented herein generate constraints that realistically model cameras and their associated intrinsic parameters and use set-based solving methods to evaluate these constraints over a 3D mesh model of a real part. This results in a complete and certifiable set of all valid camera poses describing all possible inspection poses for a given camera/part pair, as well as how much of the part's surface is inspectable from any pose in the set. These methods are tested and validated experimentally using real cameras and precise 3D tracking equipment and are shown to accurately align with real imaging results according to the hardware they are modelling for a given inspection deployment. In addition, their ability to generate full inspection solution sets is demonstrated on several realistic geometries using realistic factory settings, and they are shown to generate tangible, deployable inspection solutions, which can be readily integrated into real factory settings.

INSTANCE SEGMENTATION OF 3D MESH MODEL BY INTEGRATING 2D AND 3D DATA

Abstract. Buildings are an important part of the urban scene. In this paper, a novel instance segmentation framework for 3D mesh models in urban scenes is proposed. Unlike existing works focusing on semantic segmentation of urban scenes, this work focuses on detecting and segmenting 3D building instances even if they are attached and occluded in a large and imprecise 3D surface model. Multi-view images are first enhanced to RGBH images by adding a height map and are segmented to obtain all roof instances using Mask R-CNN. The 2D roof instances are then back-projected onto the 3D scene, the accurate 3D roof instances are obtained using a novel 3D clustering method and two post-processing steps which preserve the largest connected region and remove the model ambiguity. Finally, the 2D convex hull of each 3D roof instance is calculated and the model is divided within the range into building instances. The performance of the proposed methods is evaluated using real UAV images and the corresponding 3D mesh models qualitatively and quantitatively. Results revealed that the proposed method could effectively segment the model of the urban scenes and building instance is obtained, the over-segmentation masks can be clustered correctly into roof instances and the under-segmentation masks caused by image segmentation errors are eliminated.