3D Objects Research Articles

Tomographic projection optimization for volumetric additive manufacturing with general band constraint Lp-norm minimization

Tomographic volumetric additive manufacturing is a rapidly growing fabrication technology that enables rapid production of 3D objects through a single build step. In this process, the design of projections directly impacts geometric resolution, material properties, and manufacturing yield of the final printed part. Herein, we identify the hidden equivalent operations of three major existing projection optimization schemes and reformulate them into a general loss function where the optimization behavior can be systematically studied, and unique capabilities of the individual schemes can coalesce. The loss function formulation proposed in this study unified the optimization for binary and greyscale targets and generalized problem relaxation strategies with local tolerancing and weighting. Additionally, this formulation offers control on error sparsity and consistent dose response mapping throughout initialization, optimization, and evaluation. A parameter-sweep analysis in this study guides users in tuning optimization parameters for application-specific goals.

Multi-scale deep learning and clustering-based tabletop object instance segmentation for robot manipulation

3D object instance segmentation plays a vital role in various applications such as autonomous driving, robotics and virtual reality. However, tabletop scenes exhibit diverse object complexities and size variations. The challenge is to enhance the accuracy of segmenting these scenes for multiple object instances. This limitation directly impacts robots’ capabilities to effectively grasp and manipulate objects. In this paper, we propose a multi-scale deep learning and clustering-based approach for object instance segmentation in tabletop scenes. Our approach incorporates a multi-scale neighborhood feature sampling (MNFS) module specifically designed to extract local features, and a clustering algorithm to eliminate noise and preserve instance integrity. Furthermore, we combine the strength of both methods through ScoreNet and non-maximal suppression. We conducted extensive experiments on TO-Scene, the first large-scale dataset of 3D tabletop scenes, and observed an average mIoU improvement of approximately 4.07% compared to existing methods. This highlights the superior performance of our proposed method. In addition, we tested our algorithm on a real-scene robotics platform and showed that it has good performance and generalization capabilities to support future applications such as robot grasping.

Small Object Detection with lightweight PointNet Based on Attention Mechanisms

Abstract In response to the issue of subpar small object detection outcomes, the improved 3D object detection method for Point Net is proposed. First, multi-scale fusion method is added to the channel attention mechanism into the feature extraction stage of Point Net; then, the loss function is adjusted to optimize the backbone network and reduce the number of model parameters while improving the accuracy. The experimental results show that in the dataset, the proposed method has higher detection accuracy, the improved algorithm average accuracy improves 3.9%, the model size decreases 20%, effectively improves the detection effect of small objects, and realizes the lightweight of the network.

Single-photon-assisted two-photon polymerization

Light-based additive manufacturing (AM) has revolutionized the fabrication of complex three-dimensional objects offering a cost-effective and high-speed alternative to traditional machining. Single-photon polymerization is a key process in this advancement, providing rapid printing time, albeit with limited resolution in the range of tens of micrometers. Two-photon polymerization provides sub-micrometer resolution but is accompanied by a tradeoff of prolonged printing times. We propose combining single- and two-photon absorption to benefit from the dual capabilities, allowing for faster printing time while maintaining high resolution. In this study, we employ a continuous-wave blue light source to pre-sensitize a photocurable resin by single-photon absorption followed by a tightly focused femtosecond laser beam to provide the required energy reaching the polymerization threshold for solidifying the resin through two-photon absorption. We first investigate the impact of pre-sensitization by blue light illumination followed by irradiation with a focused femtosecond laser beam and find that the voxel growth dynamics are markedly altered. Specifically, pre-sensitization by blue light lowers the polymerization threshold power of the femtosecond laser beam and increases the speed of voxel growth. We then exploit this effect by building a custom two-photon 3D printer in which the blue light pre-sensitization is created in a light-sheet configuration. We report successfully printed 3D objects for which the average femtosecond laser power was reduced by 28.6 % and the exposure time was reduced by a factor of two compared with polymerization performed only with the femtosecond laser beam. Additionally, a 63 % improvement in axial voxel size is attained through blue light-sheet pre-sensitization. Our theoretical analysis based on a diffusion-free model suggests that the mechanism of the pre-sensitization is oxygen depletion followed by a single-photon background latent polymerization.

Transformer neural network based real-time process monitoring and direct visualization of top-down vat photopolymerization

Top-down vat photopolymerization (TVPP) technology is rapidly developing to all of the industries for new products development and manufacturing due to its low cost, fast speed and high precision. The powerful capability of TVPP for large size, highly customized and medium batch production renders it one of the most popular additive manufacturing techniques today. An effective real-time process monitoring method providing timely feedback for part defects, especially in case of print failure, is highly desirable but still rarely reported for TVPP 3D printing. Large 3D objects are normally segmented into smaller parts to reduce the risk of failure and materials waste, resulting in the complexity of building while sacrificing component integrity. Herein, a transformer neural network based real-time and visualized process monitoring (TransRV) was constructed as an effective method to enhance the manufacturing performance and quality. Upon the challenge of visualizing and capturing the real-time fabricated layer from the around liquid photoresin, a real-time dataset including in-situ standard reference images and real-time mask fabricated layers was initially constructed. Based on the dataset foundation, we then developed a novel neural network model for effective segmentation of captured images by introducing multiple attention mechanisms and adopting the architecture of Swin Transformer. The experimental results showed that the real-time taken images during the printing process could be accurately segmented through our designed neural network model. The mIoU, which is the ratio of mean intersection over union, was considered as the main evaluation index in the test set. And the value of mIoU could achieve as high as 96.14 %. On the basis of this result, we further constructed a multiple quality monitoring indicator for quality assessment and defect detection of TVPP process. It was proved that this indicator enabled real-time accurate recognition and in time feedback. The typical defects such as overall collapse and partial missing of the printed parts that usually occur during TVPP process would be timely detected and subsequently stopped printing. Apparently, the methods developed in this work provide a promising strategy to effectively eliminate the material waste and highly improve the productivity. Most importantly, the presented real-time process monitor holds the great potential for quality control and defect detection of widespread TVPP manufacturing.

Conformal printed electronics on flexible substrates and inflatable catheters using lathe-based aerosol jet printing

With the growth of additive manufacturing (AM), there has been increasing demand for fabricating conformal electronics that directly integrate with larger components to enable unique functionality. However, fabrication of conformal electronics is challenging because devices must merge with host substrates regardless of curvilinearity, topography, or substrate material. In this work, we employ aerosol jet (AJ) printing, an AM method for jet printing electronics using ink-based materials, and a custom-made lathe mechanism for mounting flexible substrates and 3D objects on a rotating axis. Using this method of lathe-based AJ printing, conformal electronics are printed around the circumference of rotational bodies with 3D curvilinear surfaces through cylindrical-coordinate motion. We characterize the diverse capabilities of lathe AJ (LAJ) printing and demonstrate flexible conformal electronics including multilayer carbon nanotube transistors. Lastly, a graphene sensor is conformally printed on an inflated catheter balloon for temperature and inflation monitoring, thus highlighting the versatilities of LAJ printing.

Real-time object detection and augmentation

This study presents the development of a real-time object detection and augmentation system using TensorFlow and Unity it leverages TensorFlow model to identify and classify objects in real-time from a camera feed, and Unity to integrate and display corresponding 3D virtual objects in an augmented reality environment. By creating a mapping system to pair detected objects with their virtual counterparts, the system aims to enhance user interaction through dynamic and immersive AR experiences. The study addresses the challenges of real-time processing and seamless integration, demonstrating the potential of combining machine learning and augmented reality to enrich interactive applications across various domains.

Radiologic evaluation of the internal carotid artery and jugular bulb in lateral temporal bone resection using 3D computed tomography.

This study investigated the internal carotid artery (ICA) and jugular bulb (JB) structures in terms of lateral temporal bone resection using 3D computed tomography (CT). We retrospectively investigated 80 ears of 40 patients using 3D reconstruction data from normal temporal bone CT. Ten critical points (P) in the temporal bone were marked in the 3D object with reference to the axial, coronal, and sagittal images of the CT scans. An imaginary plane of the facial nerve (PLf) course was also reconstructed in relation to the three points of the chorda-facial junction, P5 (second genu), and P3 (cochleariform) process. The distances (mean ± SD; mm) from points P3 to P1 (the highest level of the JB) and P2 (the posterior wall of the ascending petrous IAC at the level of the Eustachian tube) were 12.03 ± 2.56 and 9.79 ± 1.78, respectively. The distances from point P4 (chorda-facial junction) to P1 and P2 were 10.98 ± 2.70 and 17.66 ± 2.26, respectively. The angles (mean ± SD; degree) between the PLf to the line from Pa (point of the anterior bony canal) to P3 and P4 were 17.80 ± 10.05º and 8.93 ± 5.37º, respectively. The angles between the PLf to the line from P3 to P1 and P2 were - 36.35 ± 13.28º and - 24.78 ± 13.91º, respectively. The angles between the PLf to the line from P4 to P1 and P2 respectively were - 40.35 ± 15.37º and - 13.34 ± 7.63º. Understanding the anatomical relationships of P1 and P2 at P3 and P4 can be helpful in preventing iatrogenic trauma of the ICA and JB.

Iterative printing of bulk metal and polymer for additive manufacturing of multi-layer electronic circuits

In pursuing advancing additive manufacturing (AM) techniques for 3D objects, this study combines AM techniques for bulk metal and polymer on a single platform for one-stop printing of multilayer 3D electronic circuits with two novel aspects. The first innovation involves the embedded integration of electronic circuits by printing low-resistance electrical traces from bulk metal into polymer channels. Cross-section grinding results reveal (92 ± 5)% occupancy of electrically conductive traces in polymer channels despite the different thermal properties of the two materials. The second aspect encompasses the possibility of printing vertical bulk metal vias up to 10 mm in height with the potential for expansion, interconnecting electrically conductive traces embedded in different layers of the 3D object. The work provides comprehensive 3D printing design guidelines for successfully integrating fully embedded electrically conductive traces and the interconnecting vertical bulk metal vias. A smooth and continuous workflow is also introduced, enabling a single-run print of functional multilayer embedded 3D electronics. The design rules and the workflow facilitate the iterative printing of two distinct materials, each defined by unique printing temperatures and techniques. Observations indicate that conductive traces using molten metal microdroplets show a 12-fold reduction in resistance compared to nanoparticle ink-based methods, meaning this technique greatly complements multi-material additive manufacturing (MM-AM). The work presents insights into the behavior of molten metal microdroplets on a polymer substrate when printed through the MM-AM process. It explores their characteristics in two scenarios: When they are deposited side-by-side to form conductive traces and when they are deposited out-of-plane to create vertical bulk metal vias. The innovative application of MM-AM to produce multilayer embedded 3D electronics with bulk metal and polymer demonstrates significant potential for realizing the fabrication of free-form 3D electronics.

Creating High-quality 3D Content by Bridging the Gap Between Text-to-2D and Text-to-3D Generation

In recent times, automatic text-to-3D content creation has made significant progress, driven by the development of pretrained 2D diffusion models. Existing text-to-3D methods typically optimize the 3D representation to ensure that the rendered image aligns well with the given text, as evaluated by the pretrained 2D diffusion model. Nevertheless, a substantial domain gap exists between 2D images and 3D assets, primarily attributed to variations in camera-related attributes and the exclusive presence of foreground objects. Consequently, employing 2D diffusion models directly for optimizing 3D representations may lead to suboptimal outcomes. To address this issue, we present X-Dreamer, a novel approach for high-quality text-to-3D content creation that effectively bridges the gap between text-to-2D and text-to-3D synthesis. The key components of X-Dreamer are two innovative designs: Camera-Guided Low-Rank Adaptation (CG-LoRA) and Attention-Mask Alignment (AMA) Loss. CG-LoRA dynamically incorporates camera information into the pretrained diffusion models by employing camera-dependent generation for trainable parameters. This integration makes the 2D diffusion model camera-sensitive. AMA loss guides the attention map of the pretrained diffusion model using the binary mask of the 3D object, prioritizing the creation of the foreground object. This module ensures that the model focuses on generating accurate and detailed foreground objects. Extensive evaluations demonstrate the effectiveness of our proposed method compared to existing text-to-3D approaches. Our project webpage: https://anonymous-11111.github.io/ . Our code is available at https://github.com/xmu-xiaoma666/X-Dreamer .

Enhanced 3D object detection for autonomous driving: A spatial-temporal alignment approach in Bird's Eye View scenarios

This paper presents a novel 3D object detection algorithm designed for Bird's Eye View (BEV) scenarios, which significantly improves detection capabilities by integrating spatial and temporal features. The core of our approach is the spatial-temporal alignment module that efficiently processes information across different time steps and spatial locations, enhancing the precision and robustness of object detection. We employ a temporal self-attention mechanism to capture the motion information of objects over time, allowing the model to correlate features across various time steps for identifying and tracking moving objects. Additionally, a spatial cross-attention mechanism is utilized to focus on spatial features within regions of interest, promoting interactions between features extracted from camera views and BEV queries. Our method also implements temporal feature integration and multi-scale feature fusion to enhance detection stability and accuracy for fast-moving objects and to capture multi-scale context information, respectively. The model employs an enriched feature set post alignment for 3D bounding box prediction, ascertaining the position, dimensions, and orientation of objects. We conducted experiments on two public datasets for autonomous driving nuScenes and Waymo Open Dataset, demonstrating that our method outperforms previous BEVFormer and other state-of-the-art methods in terms of detection accuracy and robustness. The paper concludes with potential future directions for optimizing the BEVFormer model's performance and exploring its application in broader scenarios and tasks.

Photo/thermal dual-responsive azobenzene-based photosensitive resin for 4D printing

4D printing, combining the advantages of 3D printing and stimuli-responsive materials, provides a precise and rapid fabrication method for complex-shaped smart objects. In this study, the photoisomerization characteristic of azobenzene is used to develop 4D printable resins composed of azobenzene-based acrylates (AZO) and other acrylate compounds, which can be UV-cured and 3D printed into objects with photo and thermal dual responsiveness. UV-curing kinetics reveal that AZO competes with the photoinitiator for photons during photopolymerization, reducing the generation of reactive oxygen species and subsequently affecting the mechanical properties of UV-cured resin. The optimum content of AZO in 4D printing resins is 0.5 wt%, producing 3D printed objects with excellent mechanical properties, including a maximum stress of 25.2 MPa and a strain of 100 %. This ensures the successful formation of complex-shaped objects in the 3D printing process. Moreover, the objects printed with this photosensitive resin demonstrate exceptional shape memory capabilities, achieving a 100 % shape recovery ratio over three cycles of thermal stimulation. The printed openwork spheroid with a concave shape can successfully revert to its original form after 48 s of irradiation with 365 nm UV light. The degree of shape recovery can be precisely controlled by adjusting the UV irradiation time and position. Therefore, these photo/thermal dual-responsive 4D objects show great application prospects in medical devices and smart materials.

Experimental Determination of Tensile Stress Concentration Factors of 3D-Printed PLA Polymers: Fillet Radius, Hole Diameter, and Infill Density

3D-printing technology is being used as a regular approach in prototyping and the production of machine components. However, despite their metallic counterparts, there are many issues including infill pattern, density, and stress concentration coefficient in 3D printing that are not well-defined. The infill density plays a significant role in the printing time and mechanical properties of the printed objects. On the other hand, like metallic materials, changing geometry, such as fillet radius and hole alters the strength of the printed elements. In this work, experimental works have been conducted to determine the effect of infill density on the tensile strength of 3D printed elements. Furthermore, various standard specimens for tensile testing have been prepared to investigate the effects of fillet radius and in-plane hole diameters on the tensile strength of PLA 3D-printed elements with different infill density. Using the experimental results, the tensile stress concentration coefficients as a function of fillet radius, hole diameters, and infill density have been determined. The results of the present work can be used as a guideline for analytical design and manufacturing 3D printing objects.

Occlusion-guided multi-modal fusion for vehicle-infrastructure cooperative 3D object detection

In autonomous driving, leveraging sensor data (e.g. camera, LiDAR data) from both the vehicle and the infrastructure significantly improves perception capabilities. However, this integration traditionally results in increased demands on communication bandwidth. To address these challenges, we introduce Fusion2comm, an occlusion-guided feature fusion approach designed to optimize vehicle-infrastructure cooperative 3D object detection. Our innovative strategy employs an intelligent fusion of camera and LiDAR data to enhance the expressiveness of features. Subsequently, it leverages a segmentation model to extract foreground features and utilizes an occlusion-based selection of communication content, effectively easing bandwidth constraints. We propose a multimodal foreground feature fusion architecture that selectively processes and transmits critical information, substantially reducing irrelevant background data transfer. An innovative occlusion confidence-aware communication technique dynamically adjusts communication regions based on occlusion levels, ensuring efficient data exchange. Fusion2comm sets a new benchmark in the DAIR-V2X dataset, achieving an average precision of 71.25% with minimal bandwidth usage of 221.04 bytes. Our comprehensive experimental evaluations confirm that Fusion2comm substantially advances detection precision while simultaneously improving communication efficiency.

High-speed aerial grasping using a soft drone with onboard perception

Contrary to the stunning feats observed in birds of prey, aerial manipulation and grasping with flying robots still lack versatility and agility. Conventional approaches using rigid manipulators require precise positioning and are subject to large reaction forces at grasp, which limit performance at high speeds. The few reported examples of high-speed aerial grasping rely on motion capture systems, or fail to generalize across environments and grasp targets. We describe the first example of a soft aerial manipulator equipped with a fully onboard perception pipeline, capable of robustly localizing and grasping visually and morphologically varied objects. The proposed system features a novel passively closed tendon-actuated soft gripper that enables fast closure at grasp, while compensating for position errors, complying to the target-object morphology, and dampening reaction forces. The system includes an onboard perception pipeline that combines a neural-network-based semantic keypoint detector, a state-of-the-art robust 3D object pose estimator, and a fixed-lag smoother to estimate the pose of known objects. The resulting pose estimate is passed to a minimum-snap trajectory planner, tracked by an adaptive controller that fully compensates for the added mass of the grasped object. Finally, a finite-element-based controller determines optimal gripper configurations for grasping. Experiments on three different targets confirm that our approach enables dynamic, high-speed, and versatile grasping, all of which are necessary capabilities for tasks such as rapid package delivery or emergency relief. We demonstrate fully onboard vision-based grasps of a variety of objects, in both indoor and outdoor environments, and up to speeds of 2.0 m/s—the fastest vision-based grasp reported in the literature. Finally, we take a major step in expanding the utility of our platform beyond stationary targets, by demonstrating motion-capture-based grasps of targets moving up to 0.3 m/s, with relative speeds up to 1.5 m/s.

Study on the Development of a Virtual Reality Application for Skeletal Anatomy Learning Using Rapid Application Development (RAD)

The use of innovative multimedia technology continues to evolve and has now become a primary technology tool in education. Currently, the utilization of virtual reality (VR) technology is being used as an immersive medium for learning, especially in the field of medicine. Immersive technology is presented in 3D visualization, approaching real and relevant objects in medical education as an alternative to traditional teaching aids such as cadavers. In this research, an exploration of the VR environment was conducted in a case study of the skeletal system using 3D skull bone objects. The exploration involved tasks such as preparing the 3D skull bone object data assets in the 3D development environment, exporting data assets to the 3D-VR environment, and configuring aspects related to visualization and object control based on visual C#. So far, documentation of the 3D-VR application development environment and the trial results of the case study object development on the Unity platform have been produced. Additionally, documentation of the interface design application using Rapid Application Development (RAD) has also been generated. The Android-based VR application has functioned well in accommodating the skull object, designed with 3 layers of the skeletal system adjusted to their respective depths. The skull consists of 22 components, including cranium.

Deep learning for 3D object recognition: A survey

With the growing availability of extensive 3D datasets and the rapid progress in computational power, deep learning (DL) has emerged as a highly promising approach for learning from 3D data, addressing critical tasks like object detection, segmentation, and recognition. Despite the unique challenges in processing geometry data with deep neural networks, recent advancements in DL for 3D object recognition have shown remarkable success, with various methods proposed to tackle different issues. This paper aims to stimulate future research by providing a comprehensive review of recent progress in DL techniques for 3D object recognition, which are systematically categorized based on their learning behavior. We discuss the advantages, limitations, and application of each approach, highlighting their performance in 3D object classification on benchmark datasets such as ModelNet, ScanObjectNN, and Sydney Urban Object. The survey offers insightful observations and inspires future research directions.

OpenECAD: An efficient visual language model for editable 3D-CAD design

Computer-aided design (CAD) tools are utilized in the manufacturing industry for modeling everything from cups to spacecraft. These programs are complex to use and typically require years of training and experience to master. Structured and well-constrained 2D sketches and 3D constructions are crucial components of CAD modeling. A well-executed CAD model can be seamlessly integrated into the manufacturing process, thereby enhancing production efficiency. Deep generative models of 3D shapes and 3D object reconstruction models have garnered significant research interest. However, most of these models produce discrete forms of 3D objects that are not editable. Moreover, the few models based on CAD operations often have substantial input restrictions. In this work, we fine-tuned pre-trained models to create OpenECAD models (0.55B, 0.89B, 2.4B and 3.1B), leveraging the visual, logical, coding, and general capabilities of visual language models. OpenECAD models can process images of 3D designs as input and generate highly structured 2D sketches and 3D construction commands, ensuring that the designs are editable. These outputs can be directly used with existing CAD tools’ APIs to generate project files. To train our network, we created a series of OpenECAD datasets. These datasets are derived from existing public CAD datasets, adjusted and augmented to meet the specific requirements of vision language model (VLM) training. Additionally, we have introduced an approach that utilizes dependency relationships to define and generate sketches, further enriching the content and functionality of the datasets.

Radar-camera fusion for 3D object detection with aggregation transformer

Radar-camera fusion for 3D object detection with aggregation transformer

A Systematic Survey of Transformer-Based 3D Object Detection for Autonomous Driving: Methods, Challenges and Trends

A Systematic Survey of Transformer-Based 3D Object Detection for Autonomous Driving: Methods, Challenges and Trends