CAD-RAG: A multi-modal retrieval augmented framework for user editable 3D CAD model generation

As Virtual Reality(VR), Augmented Reality(AR), Mixed Reality(MR) technology becomes more accessible, it is important to explore VR/AR/MR technologies that can be used for remote collaboration on physical tasks. Previous research has shown that gesture-based interaction is intuitive and expressive for remote collaboration, and using 3D CAD models can provide clear instructions for assembly tasks. In this paper, therefore, we describe a new MR remote collaboration system which combines the use of gesture and CAD models in a complementary manner. The prototype system enables a remote expert in VR to provide instructions based on 3D gesture and CAD models (3DGAM) for a local worker who uses AR to see these instructions. Using this interface, we conducted a formal user study to explore the effect of sharing 3D gesture and CAD models in an assembly training task. We found that the combination of 3D gesture and CAD models can improve remote collaboration on an assembly task with respect to the performance time and user experience. Finally, we provide some conclusions and directions for future research.

Data registration between 3D point cloud and CAD model of non-cooperative object has been considered as one of key technologies for estimating spatial position and orientation of target spacecraft. The registration result will directly affect the success or failure of on-orbit capture mission for space manipulator. Usually, 3D CAD model needs to discretize into point cloud of model which can be applied to match the corresponding 3D measuring point clouds. In this article, a coarse registration algorithm of curvature features based on distance constraint consistency is proposed to solve data registration between 3D point cloud and CAD model of non-cooperative object. According to the principle of invariant curvature of rigid transformation, a set of curvature feature points which satisfies the consistency of distance constraint can be selected to calculate rotation matrix and translation vector between both sets of point clouds. Experimental results have shown that the proposed registration algorithm can achieve higher registration accuracy to provide reliable initial values of transformation parameters for subsequent fine registration work.

Inaccurate results are usually produced due to lack of precise dynamic parameters when analyzing the running stability of railway vehicles. In this paper, a new solution is proposed to conduct the running stability analysis of railway vehicles supported by their three-dimensional (3D) CAD models, which are realized based on the intrinsic mapping relationship between the dynamic model and the 3D CAD model of railway vehicles. First, a dynamic model library of railway vehicles is established, and for each model type the software package for running stability analysis is programmed with Matlab. Then according to the requested parameters of each model, the corresponding dynamic parameter extraction flow based on 3D CAD model is designed by the secondary development of CAD software. The user chooses CAD objects and sets part of dynamic parameters using a wizard-style guiding dialogs; the background program will automatically and accurately extract the physical and geometric parameters from the CAD model, and then call the Matlab software package and transfer the parameter values, with analysis results presented in CAD environment. The case study shows that, this method can not only ensure the precision of dynamic parameters, but also improve the dynamic modeling efficiency for running stability analysis of railway vehicles.

A Component-Based Approach for Generation of 3 Dimensional CAD Models Using a Construction MODEL Component Database Hung-Ming Chen and Yi-Chi Chu Pages 460-467 (2009 Proceedings of the 26th ISARC, Austin, USA, ISBN 978-0-578-02312-0, ISSN 2413-5844) Abstract: In recent years, three-dimensional visualization technologies have been researched and applied for aiding the operations of design, planning, construction, etc. in construction engineering. A three-dimensional (3D) CAD model of a targeted facility or construction project is a requirement. Due to the uniqueness and custom features of construction products, their 3D models usually have to be built based on basic geometric modeling elements from scratch. Generally, this is a time-consuming and labor-intensive process. In addition, the reusability of built 3D models is low. For improving efficiency in building 3D models and reusability of built 3D models, this research proposes a component-based approach for generation of 3D models using a construction model component database. A construction model component database is developed to store and provide 3D models of construction components, such as materials and equipments, to be queried and retrieved for generating global 3D models in construction engineering. A management interface of the database is developed for inserting and editing construction component models. A model building interface which utilizes the database for generating global 3D models based on search, selection, and then setting operation is also developed. The data format of 3D component models in the database is X3D for interoperability. Base on the proposed approach, 3D CAD models can be built more efficiently by instancing existing components in the database. In addition, built models can be decomposed into component models to be stored in the database for being reused. Keywords: database; Visualization; 3D model; CAD; Virtual Construction DOI: https://doi.org/10.22260/ISARC2009/0003 Download fulltext Download BibTex Download Endnote (RIS) TeX Import to Mendeley

Introduction: Sarcomas are rare connective tissue tumors with over 100 described subtypes. High heterogeneity within and among sarcoma sub-entities and low incidence contribute to poor therapeutic success. Patient-derived 3D models (PDMs), i.e. organoids and spheroids, closely recapitulate features of the original tumor, e.g. cell-to-cell interaction, differentiation potential and heterogeneity. However, PDMs are rarely available or even lacking for most sarcoma sub-entities, constituting a major hurdle for sarcoma precision oncology and research. Here, we present a highly efficient and systematic framework to generate long-term 3D models for distinct sarcoma subtypes, which we developed within the HEROES-AYA consortium. Methods: Tumors were received directly after surgery and processed within 24 hours. After enzymatic dissociation, cell clumps were seeded in 384-well plates with and without Matrigel. Each tumor was cultivated on an array of 140 distinct culture conditions, differing in contribution and concentration of pathway inhibitors and activators. Organoid and spheroid growth was assessed over time via imaging and AI-based algorithms to identify best-growing conditions. Results: Initially, we combined Ewing sarcoma (EwS) secreted factors and potentially relevant cytokines to generate a matrix of 140 diverse media, named Media Test Plate (MTP). The MTP was tested on five EwS tumors, allowing the identification of optimal culture conditions with 100% efficiency. When transplanted in NSG mice, the established PDMs showed tumorigenicity potential and Ewing sarcoma-specific CD99 marker expression. The MTP was then tested on four synovial sarcomas (SyS) and five myxoid liposarcomas with a 75% and 50% success rate, respectively. The optimal media, as identified by MTP, allowed the cultivation of one EwS and one SyS out of two, respectively, even with limited cell count. Additional PDMs generated include one undifferentiated, one alveolar soft tissue and one spindle cell sarcoma. Established cultures were characterized by Ki67 positivity, ranging from 40% to 70%, and expression of sarcoma subtype-specific markers. EwS models resulted 100% CD99 positive, while TLE1 positivity in SyS models was ≈90%. DNA and RNA sequencing of PDMs will show whether they resemble the genetic and transcriptional features of the original tumors. Conclusions: We here present a novel systematic protocol to generate organoids and spheroids from different sarcoma subtypes. To date, no such systematic protocol is available for sarcoma models generation. Our success rate is comparable to the one of intestinal and pancreatic organoids and exceeds those of 2D and PDX sarcoma cultures, which is ≈20%. Current work aims to adapt the MTP to a broad range of sarcoma subtypes, including rare entities. The efficient generation of expandable PDMs from a wide range of sarcomas will further fuel basic and translational research. Citation Format: Claudia Dagostino, Waqar Hussain, Jasmina Paluncic, Marcelo Zoccoler, Jessica Pablik, Daniela Richter, Lisanne Knol, Ana Banito, Johanna Wagner, Ivona Mateska, Daniel E. Stange, Ina Oehme, Stephan Richter, Stefan M. Pfister, Stefan Fröhling, Claudia Scholl, Jürgen Weitz, Klaus-Dieter Schaser, Christoph Heining, Hanno Glimm, Claudia R. Ball. A systematic framework for efficient generation of expandable patient-derived 3D sarcoma models [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4255.

A framework for synthetic image generation and augmentation for improving automatic sewer pipe defect detection

A framework for prismatic part-data generation based on the concept of the unit-machined loop is proposed. The unit-machine loop is a removing volume that can be machined with a type of cutter within a type of operation. With the aid of this concept, the traditional feature representation model is discarded, and the process reasoning logic development is simplified. The factors of critical information and process knowledge of the removing volumes which influence the unit-machined loop construction are discussed. The input of the framework can be either from a two-dimensional computer-aided-design system or from a three-dimensional solid modelling system. The interface algorithms of two- and three-dimensional systems are presented. Finally, the rules for constructing the unit-machined loops are given and the concept is illustrated by the use of examples.

Microscopic simulations may bring a better understanding of the response of gaseous detectors. Such simulations are computationally demanding, due to the modelling of the low energy processes and to the high segmentation required for the 2D/3D field maps. In MPGD such maps can be much more complex than those of traditional multiwire chambers, due to the heterogeneous materials and more involute geometries, which break the simplifying symmetries featured in the latter. In order to investigate the performance of the triple GEM 2-dimensional tracking chambers being developed for high luminosity experiments with the Super BigBite Spectrometer at Jefferson Laboratory, we have set up a flexible and rather effcient multistep simulation processor based on either ANSYS or GMSH+ELMER for 3D CAD and electrostatic field modelling and then combined to Garfield++. Potential systematic effects from the 3D CAD modellers, the mesh generators and the electrostatic field solvers have been estimated with dedicated simulations; once these effects have been assessed, the results of the multistep approach have been compared to a simplified whole GEM chamber model.

The most common way to retrieve symmetry information (i.e., the planes and axes of symmetry) in 3D CAD models is through visual recognition by engineers. However, engineers are not able to visually recognize exact symmetry in any CAD model, and their ability to recognize symmetry decreases as the number of CAD models increases. To overcome these limitations, computer-aided symmetry detection is employed, which enables the (semi)automatic extraction of the symmetry information in CAD models. Hence, the present paper introduces a symmetry detection framework for 3D CAD models with boundary representation. The novelty of this research was that it addressed the detection of exact and partial axi- and reflectional symmetry in CAD models with analytic and numeric surfaces. Further, symmetry measures were proposed to differentiate exact, partial, or non-symmetry in the CAD model. The framework was implemented into a state-of-the-art CAD system and subjected to performance and time complexity validation. The results showed that the implemented framework’s performance was 0.94 F1-score, and the time complexity was linear with respect to the number of faces in the CAD model. Hence, it was concluded that the framework is suitable for industrial applications to support engineers in symmetry detection.

When performing non-destructive testing of aerostructures with complex geometrical shapes and curved surfaces, two problems arise: one is generation of an appropriate sequence of 3D coordinates and orientations for the inspection probe to follow complex component contours; the other is interpretation and characterιsation of defects based on 2D NDT images. By utilising the information from CAD models, this paper presents a 3D NDT software package to aid planning of NDT data acquisition via 3D simulation, and to aid defect visualisation and measurement in 3D by superimposing 2D NDT images on their 3D CAD models. The particular noteworthy and unique characteristics of the software are (a) a versatile 3D CAD model editing tool for extraction of CAD features and construction of simplified CAD models; (b) 3D scanning simulation for automatic generation of probe coordinates and orientations with respect to curved component surfaces; (c) tools for aligning and projecting a 3D CAD model on to its 2D NDT image; and (d) texture mapping to project a NDT image on to its 3D CAD model surface. Using real aircraft components with their radiographic, ultrasonic and thermographic images as examples, the software is shown to offer a unique, versatile and powerful CAD-based 3D NDT environment for fast and reliable structure integrity assessment of complex aerostructures.

During the new product development process, reusing the existing CAD models could avoid designing from scratch and decrease human cost. With the advent of big data, how to rapidly and efficiently find out suitable 3D CAD models for design reuse is taken more attention. Currently the sketch-based retrieval approach makes search more convenient, but its accuracy is not high enough; on the other hand, the semantic-based retrieval approach fully utilizes high level semantic information, and makes search much closer to engineers’ intent. However, effectively extracting and representing semantic information from data sets is difficult. Aiming at these problems, we proposed a sketch-based semantic retrieval approach for reusing 3D CAD models. Firstly a fine granularity semantic descriptor is designed for representing 3D CAD models; Secondly, several heuristic rules are adopted to recognize 3D features from 2D sketch, and the correspondences between 3D feature and 2D loops are built; Finally, semantic and shape similarity measurements are combined together to match the input sketch to 3D CAD models. Hence the retrieval accuracy is improved. A sketch-based prototype system is developed. Experimental results validate the feasibility and effectiveness of our proposed approach.

3D CAD model matching from 2D local invariant features

At the conceptual product design stage, designers prefer sketching initial design ideas on paper before using more sophisticated 3D CAD systems for their creativity and flexible design changes. At the later detailed design stages, however, precise 3D CAD modeling is a better choice for design analyses and downstream manufacturing applications. For an integrated digital product development, the 2D drawings need to be converted to 3D CAD models manually through some laborious and time-consuming processes. For decades, many researchers have contributed to the studies on reconstructing 3D geometric models from 2D sketches, but few describe constructing parametric feature-based CAD models that are currently widely used in the detailed design stages. In this paper, a procedure for constructing a feature-based 3D CAD model directly from a single-view 2D drawing is presented. The input sketch is assumed to be an exact orthogonal projection of a 3D model viewed at a general orientation. The model geometry is limited such that it can be constructed by a Boolean combination of extrusions of polygonal profiles.

3D digitization is the process of developing a CAD model and a manufacturing database from an existing part. This process is appealing because it could be difficult to create models of complex objects using CAD software without the aid of a 3D digitizer. The 3D CAD models that are generated from the scanned data can be edited and modified in the CAD system. The 3D model can also be put in multimedia or animation software as a learning or assembly aid. Rapid prototyping is widely known as being able to fabricate 3D objects with complex geometric shapes. Rapid prototyping can shorten the product development cycle and improve the design process by providing rapid and effective feedback to the designer, thus helping a company to keep ahead in a rapidly changing market. In the application of concurrent engineering and reverse engineering, the integration of 3D digitization and rapid prototyping can greatly shorten the product development cycle time. The 3D CAD model from the 3D digitizer can be modified by CAD software and the prototype can be manufactured using rapid prototyping to save lots of time and money. In a 3D digitizing system, we use a 3D laser scanner - Modelmaker - which combines a manually-operated arm with 6 degrees of freedom, a stripe 3D laser scanner and a contact probe. This fast and flexible 3D digitizer can produce complex models. By using post-processing software and a CAD system, an STL file can be produced, so we can produce a physical part by using the rapid prototyping system. In addition, this paper presents software and hardware technologies to build an LCD panel display-based rapid prototyping system. Software design for the system includes a slicing algorithm, LCD photomask display process, user interface and motion control program. The hardware configuration of the architecture includes an LCD photomask, optical system, z-axis elevator, resin supply system and PC-based control system.

FreeCAD: A Multimodal Framework for 3D CAD Model Generation from Free-Form Prompts