3D Target Research Articles

Research and Implementation of 3D Object Detection Based on Autonomous Driving Scenarios

Perception function, as an important part of autonomous driving, ensures the safety and intelligence of driving. We collect the surrounding environment of driving through hardware sensors, providing a basis for subsequent decision-making of autonomous driving. In recent years, deep learning has made breakthrough progress in object detection. Based on this, this paper uses lidar point cloud data, combined with deep learning theory and method, to carry out three-dimensional target detection tasks around lidar point cloud data, and carries out theoretical analysis, method verification, and result analysis. A 3D object detection method based on LiDAR point cloud data is proposed. The backbone network of the network model corresponding to the method is VoxelNet’s backbone network. After outputting the feature matrix, sparse point clouds are supplemented with point clouds, and then the feature matrix is decoded to generate candidate boxes. The model used in this paper on the KITTI data set can effectively solve the problem of 3D target detection in the process of autonomous driving and has good performance.

Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving.

This paper proposes a multimodal fusion 3D target detection algorithm based on the attention mechanism to improve the performance of 3D target detection. The algorithm utilizes point cloud data and information from the camera. For image feature extraction, the ResNet50 + FPN architecture extracts features at four levels. Point cloud feature extraction employs the voxel method and FCN to extract point and voxel features. The fusion of image and point cloud features is achieved through regional point fusion and voxel fusion methods. After information fusion, the Coordinate and SimAM attention mechanisms extract fusion features at a deep level. The algorithm's performance is evaluated using the DAIR-V2X dataset. The results show that compared to the Part-A2 algorithm; the proposed algorithm improves the mAP value by 7.9% in the BEV view and 7.8% in the 3D view at IOU = 0.5 (cars) and IOU = 0.25 (pedestrians and cyclists). At IOU = 0.7 (cars) and IOU = 0.5 (pedestrians and cyclists), the mAP value of the SECOND algorithm is improved by 5.4% in the BEV view and 4.3% in the 3D view, compared to other comparison algorithms.

Efficient physics-based learned reconstruction methods for real-time 3D near-field MIMO radar imaging

Near-field multiple-input multiple-output (MIMO) radar imaging systems have recently gained significant attention. These systems generally reconstruct the three-dimensional (3D) complex-valued reflectivity distribution of the scene using sparse measurements. Consequently, imaging quality highly relies on the image reconstruction approach. Existing analytical reconstruction approaches suffer from either high computational cost or low image quality. In this paper, we develop novel non-iterative deep learning-based reconstruction methods for real-time near-field MIMO imaging. The goal is to achieve high image quality with low computational cost at compressive settings. The developed approaches have two stages. In the first approach, physics-based initial stage performs adjoint operation to back-project the measurements to the image-space, and deep neural network (DNN)-based second stage converts the 3D backprojected measurements to a magnitude-only reflectivity image. Since scene reflectivities often have random phase, DNN processes directly the magnitude of the adjoint result. As DNN, 3D U-Net is used to jointly exploit range and cross-range correlations. To comparatively evaluate the significance of exploiting physics in a learning-based approach, two additional approaches that replace the physics-based first stage with fully connected layers are also developed as purely learning-based methods. The performance is also analyzed by changing the DNN architecture for the second stage to include complex-valued processing (instead of magnitude-only processing), 2D convolution kernels (instead of 3D), and ResNet architecture (instead of U-Net). Moreover, we develop a synthesizer to generate large-scale dataset for training the neural networks with 3D extended targets. We illustrate the performance through experimental data and extensive simulations. The results show the effectiveness of the developed physics-based learned reconstruction approach compared to commonly used approaches in terms of both runtime and image quality at highly compressive settings. Our source codes and dataset are made available at https://github.com/METU-SPACE-Lab/Efficient-Learned-3D-Near-Field-MIMO-Imaging upon publication to advance research in this field.

Doped nanomaterial facilitates 3D printing target plate for rapid detection of alkaloids in laser desorption/ionization mass spectrometry.

This work aims to rapidly detect toxic alkaloids in traditional Chinese medicines (TCM) using laser desorption ionization mass spectrometry (LDI-MS). We systematically investigated twelve nanomaterials (NMs) as matrices and found that MoS2 and defect-rich-WO3 (D-WO3) were the best NMs for alkaloid detection. MoS2 and D-WO3 can be used directly as matrices dipped onto conventional ground steel target plates. Additionally, they can be conveniently fabricated as three-dimensional (3D) NM plates, where the MoS2 or D-WO3 NM is doped into resin and formed using a 3D printing process. We obtained good quantification of alkaloids using a chemothermal compound as an internal standard and detected related alkaloids in TCM extracts, Fuzi (Aconiti Lateralis Radix Praeparata), Caowu (Aconiti Kusnezoffii Radix), Chuanwu (Aconiti Radix), and Houpo (Magnoliae Officinalis Cortex). The work enabled the advantageous "dip and measure" method, demonstrating a simple and fast LDI-MS approach that achieves clean backgrounds for alkaloid detection. The 3D NM plates also facilitated mass spectrometry imaging of alkaloids in TCMs. This method has potential practical applications in medicine and food safety. Doped nanomaterial facilitates 3D printing target plate for rapid detection of alkaloids in laser desorption/ionization mass spectrometry.

Retracted: Multiview Fusion 3D Target Information Perception Model in Nighttime Unmanned Intelligent Vehicles

Retracted: Multiview Fusion 3D Target Information Perception Model in Nighttime Unmanned Intelligent Vehicles

A 2D image 3D reconstruction function adaptive denoising algorithm.

To address the issue of image denoising algorithms blurring image details during the denoising process, we propose an adaptive denoising algorithm for the 3D reconstruction of 2D images. This algorithm takes into account the inherent visual characteristics of human eyes and divides the image into regions based on the entropy value of each region. The background region is subject to threshold denoising, while the target region undergoes processing using an adversarial generative network. This network effectively handles 2D target images with noise and generates a 3D model of the target. The proposed algorithm aims to enhance the noise immunity of 2D images during the 3D reconstruction process and ensure that the constructed 3D target model better preserves the original image's detailed information. Through experimental testing on 2D images and real pedestrian videos contaminated with noise, our algorithm demonstrates stable preservation of image details. The reconstruction effect is evaluated in terms of noise reduction and the fidelity of the 3D model to the original target. The results show an average noise reduction exceeding 95% while effectively retaining most of the target's feature information in the original image. In summary, our proposed adaptive denoising algorithm improves the 3D reconstruction process by preserving image details that are often compromised by conventional denoising techniques. This has significant implications for enhancing image quality and maintaining target information fidelity in 3D models, providing a promising approach for addressing the challenges associated with noise reduction in 2D images during 3D reconstruction.

Real Time Volumetric MRI for MR-Guided 3D Motion Tracking via Sparse Prior-Augmented Neural Representation Learning

To reconstruct volumetric MRI from orthogonal cine acquisition aided by sparse priors of 2 static 3D MRI through implicit neural representation (NeRP) learning, with the goal of eliminating large-scale training datasets for data-driven sparse MRI reconstruction and supporting clinical workflow of real time 3D motion tracking during MR-guided radiotherapy. A multi-layer perceptron network was trained to learn the NeRP of a patient-specific MRI dataset, where the network takes 4D data coordinates of voxel locations and motion states as inputs and outputs corresponding voxel intensities. By first learning the NeRP of 2 static 3D MRI with different breathing motion states, prior knowledge of patient breathing motion was embedded into network weights through optimization. The prior knowledge was then augmented from 2 to 31 motion states by querying the optimized network at interpolated/extrapolated motion state coordinates. Starting from the prior-augmented network as an initialization point, the network was further trained using sparse samples of 2 orthogonal cine slices. The final volumetric reconstruction was obtained by querying the trained network at desired 3D spatial locations. We evaluated the proposed method using 5-minute volumetric MRI time series with 340 ms temporal resolution collected from 7 liver carcinoma patients. The time series was acquired using golden-angle radial MRI sequence and reconstructed through retrospective sorting. Two MRI with inhale and exhale states respectively were selected from the first 30 sec of the time series for prior embedding and augmentation. The remaining 4.5-min time series was used for volumetric reconstruction evaluation, where we retrospectively subsampled each MRI to 2 orthogonal slices and compared network-reconstructed images to ground truth images in terms of image quality and the capability of supporting 3D target motion tracking. Across the 7 patients evaluated, the peak signal to noise ratio between model reconstruction and ground truth was 54.66 ± 6.16 dB and the structural similarity index measure was 0.99 ± 0.01. Gross tumor volume (GTV) contours estimated by deforming a reference state MRI to model-reconstructed and ground truth MRI showed good consistency. The 95-percentile Hausdorff distance between GTV contours was 1.89 ± 1.13 mm, which is less than the voxel dimension. The mean GTV centroid position difference between ground truth and model estimation was less than 1 mm in all 3 orthogonal directions. Volumetric MRI from orthogonal cine acquisition with sparse priors is feasible by modeling prior knowledge through implicit neural representation learning. The model-reconstructed images showed sufficient accuracy in supporting 3D motion tracking of abdominal targets. By eliminating the need for large scale training datasets, the method promises to enable clinical implementation of 3D motion tracking for precision radiation therapy.

Image-free target identification using a single-point single-photon LiDAR.

Single-photon light detection and ranging (LiDAR) - offering single-photon sensitivity and picosecond temporal resolution - has become one of the most promising technologies for 3D imaging and target detection. Generally, target detection and identification requires the construction of an image, performed by a raster-scanned or an array-based LiDAR system. In contrast, we demonstrate an image-free target identification approach based on a single-point single-photon LiDAR. The idea is to identify the object from the temporal data equipped with an efficient neural network. Specifically, the target is flood-illuminated by a pulsed laser and a single-point single-photon detector is used to record the time-of-flight (ToF) of back-scattering photons. A deep-learning method is then employed to analyze the ToF data and perform the identification task. Simulations with indoor and outdoor experiments show that our approach can identify the class and pose of the target with high accuracy. Importantly, we construct a compact single-point single-photon LiDAR system and demonstrate the practical capability to identify the types and poses of drones in outdoor environments over hundreds of meters. We believe our approach will be useful in applications for sensing dynamic targets with low-power optical detection.

Spine Stereotactic Body Radiation Therapy Without Immobilization: Detailed Analysis of Intrafraction Motion Using High-Frequency kV Imaging During Irradiation

Spine SBRT requires high positioning accuracy and a stable patient to maximize target coverage and reduce excessive irradiation to organs at risk (OARs). Positional verification during spine SBRT delivery helps to ensure accurate positioning for all patients. We report our experience with non-invasive 3-dimensional (3D) target position monitoring during volumetric modulated arc therapy (VMAT) of spine metastases, in non-immobilized patients positioned using only a thin mattress and simple arm and knee supports. Fluoroscopic planar kV images were acquired at 7 frames/second using the on-board imaging system during VMAT spine SBRT. Template matching and triangulation were used to track the target in vertical, longitudinal and lateral direction. If the tracking trace deviated >1mm from the planned position in ≥1 direction, treatment was manually interrupted and 6D CBCT-based couch correction was performed. Tracking data was used to retrospectively analyze the target position. Positional data, agreement with CBCT, correlation between position of the couch and direction of any positional correction, and treatment times were analyzed. 175 fractions were analyzed. Delivery was interrupted 83 times in 66 fractions for a deviation >1mm. In 97% of cases the difference between tracking data and subsequent clinical shift performed after the CBCT match was ≤0.5mm. Lateral/longitudinal shift performed after intervention correlated with the couch roll/pitch at the start of treatment (correlation coefficient: -0.63/0.53). Mean (SD; range) time between start of first imaging and end of the last arc was 15.2 minutes (5.1;7.6-36.3). Spine tracking during irradiation can be used to prompt an intervention CBCT scan and repositioning, so that a spine SBRT target deviates by ≤1mm from the planned position, even in non-immobilized patients. kV tracking and CBCT are in good agreement. The data support verification CBCT after all 6 degrees-of-freedom positional corrections in non-immobilized spine SBRT patients.

A High-Precision 3D Target Perception Algorithm Based on a Mobile RFID Reader and Double Tags

With the popularization of positioning technology, more and more industries have begun to pay attention to the application and demand of location information, and almost all industries can benefit from low-cost and high-precision location information. This paper introduces a novel three-dimensional (3D) low-cost, high-precision target perception algorithm that utilizes a Radio Frequency Identification (RFID) mobile reader and double tags. Initially, the Received Signal Strength (RSS) is employed to estimate the approximate position of the target along the length direction of the shelf. Additionally, double tags are affixed to the target, enabling the perception of its approximate height and depth through phase information measurements. Subsequently, the obtained rough position serves as an initial value for calibration using the proposed algorithm, allowing for the refinement of the target’s length information relative to the shelf. Simulation results demonstrate the exceptional accuracy of the proposed method in perceiving the 3D position information of the target, achieving centimeter-level sensing accuracy.

An application of stereo matching algorithm based on transfer learning on robots in multiple scenes

Robot vision technology based on binocular vision holds tremendous potential for development in various fields, including 3D scene reconstruction, target detection, and autonomous driving. However, current binocular vision methods used in robotics engineering have limitations such as high costs, complex algorithms, and low reliability of the generated disparity map in different scenes. To overcome these challenges, a cross-domain stereo matching algorithm for binocular vision based on transfer learning was proposed in this paper, named Cross-Domain Adaptation and Transfer Learning Network (Ct-Net), which has shown valuable results in multiple robot scenes. First, this paper introduces a General Feature Extractor to extract rich general feature information for domain adaptive stereo matching tasks. Then, a feature adapter is used to adapt the general features to the stereo matching network. Furthermore, a Domain Adaptive Cost Optimization Module is designed to optimize the matching cost. A disparity score prediction module was also embedded to adaptively adjust the search range of disparity and optimize the cost distribution. The overall framework was trained using a phased strategy, and ablation experiments were conducted to verify the effectiveness of the training strategy. Compared with the prototype PSMNet, on KITTI 2015 benchmark, the 3PE-fg of Ct-Net in all regions and non-occluded regions decreased by 19.3 and 21.1% respectively, meanwhile, on the Middlebury dataset, the proposed algorithm improves the sample error rate at least 28.4%, which is the Staircase sample. The quantitative and qualitative results obtained from Middlebury, Apollo, and other datasets demonstrate that Ct-Net significantly improves the cross-domain performance of stereo matching. Stereo matching experiments in real-world scenes have shown that it can effectively address visual tasks in multiple scenes.

Development of a prediction model for target positioning by using diaphragm waveforms extracted from CBCT projection images.

To develop a prediction model (PM) for target positioning using diaphragm waveforms extracted from CBCT projection images. Nineteen patients with lung cancer underwent orthogonal rotational kV x-ray imaging lasting 70 s. IR markers placed on their abdominal surfaces and an implanted gold marker located nearest to the tumor were considered as external surrogates and the target, respectively. Four different types of regression-based PM were trained using surrogate motions and target positions for the first 60 s, as follows: Scenario A: Based on the clinical scenario, 3D target positions extracted from projection images were used as they were (PMCL ). Scenario B: The short-arc 4D-CBCT waveform exhibiting eight target positions was obtained by averaging the target positions in Scenario A. The waveform was repeated for 60 s (W4D-CBCT ) by adapting to the respiratory phase of the external surrogate. W4D-CBCT was used as the target positions (PM4D-CBCT ). Scenario C: The Amsterdam Shroud (AS) signal, which depicted the diaphragm motion in the superior-inferior direction was extracted from the orthogonal projection images. The amplitude and phase of W4D-CBCT were corrected based on the AS signal. The AS-corrected W4D-CBCT was used as the target positions (PMAS-4D-CBCT ). Scenario D: The AS signal was extracted from single projection images. Other processes were the same as in Scenario C. The prediction errors were calculated for the remaining 10 s. The 3D prediction error within 3mm was 77.3% for PM4D-CBCT , which was 12.8% lower than that for PMCL . Using the diaphragm waveforms, the percentage of errors within 3mm improved by approximately 7% to 84.0%-85.3% for PMAS-4D-CBCT in Scenarios C and D, respectively. Statistically significant differences were observed between the prediction errors of PM4D-CBCT and PMAS-4D-CBCT . PMAS-4D-CBCT outperformed PM4D-CBCT , proving the efficacy of the AS signal-based correction. PMAS-4D-CBCT would make it possible to predict target positions from 4D-CBCT images without gold markers.

Fusion Architectures for 3D Target Tracking Using IRST and Radar Measurements

Seven different architectures are presented to fuse IRST and radar data to track the target in 3D Cartesian coordinates, with the measurements available in polar coordinates. Performance of these architectures is checked with simulated data. Detailed mathematical expressions are provided which could be useful for algorithm implementation. From this study, it is concluded that CM (Common Measurements) architecture gives state estimates with relatively less uncertainty followed by SVF (State Vector Fusion). IRST gives target states with relatively high uncertainty followed by radar. This shows the necessity of the fusion in tracking system. In all, CM architecture is very simple, easy to implement and can be used in real time.

Monocular 3D Object Detection Using Depth Fusion

It is an important task to estimate a 3D bounding box from monocular images for autonomous driving. However, the monocular pictures do not have distance information, so it is difficult to acquire accurate results. For the sake of solving the trouble of low accuracy of the monocular image in 3D target detection because of lacking distance information, an improved monocular three-dimensional target detection algorithm based on GUPNet and neural network was proposed to promote the precision of target detection. First, based on the geometric method proposed by GUPNet, the depth, and uncertainty are obtained by direct regression using a neural network. According to the difference in the accuracy of the two methods, a parameter α was introduced, and their depth scores are obtained from the uncertainty. According to the depth score and parameter α, the depth obtained by the two methods is fused to get the final depth. Test results prove that the proposed algorithm promotes average detection precision of KITTI data set in simple, medium, and difficult cases.

Inverse design and additive manufacturing of shape-morphing structures based on functionally graded composites

Shape-morphing structures possess the ability to change their shapes from one state to another, and therefore, offer great potential for a broad range of applications. A typical paradigm of morphing is transforming from an initial two-dimensional (2D) flat configuration into a three-dimensional (3D) target structure. One popular fabrication method for these structures involves programming cuts in specific locations of a thin sheet material (i.e. kirigami), forming a desired 3D shape upon application of external mechanical load. By adopting the non-linear beam equation, an inverse design strategy has been proposed to determine the 2D cutting patterns required to achieve an axisymmetric 3D target shape. Specifically, tailoring the localised variation of bending stiffness is the key requirement. In this paper, a novel inverse design strategy is proposed by modifying the bending stiffness via introducing distributed modulus in functionally graded composites (FGCs). To fabricate the FGC-based shape-morphing structures, we use a multi-material 3D printer to print graded composites with voxel-like building blocks. The longitudinal modulus of each cross-sectional slice can be controlled through the rule of mixtures according to the micro-mechanics model, hence matching the required modulus distribution along the elastic strip. Following the proposed framework, a diverse range of structures is obtained with different Gaussian curvatures in both numerical simulations and experiments. A very good agreement is achieved between the measured shapes of morphed structures and the targets. In addition, the compressive rigidity and specific energy absorption during compression of FGC-based hemi-ellipsoidal morphing structures with various aspect ratios were also examined numerically and validated against experiments. By conducting systematical numerical simulations, we also demonstrate the multifunctionality of the modulus-graded shape-morphing composites. For example, they are capable of blending the distinct advantages of two different materials, i.e. one with high thermal (but low electrical) conductivity, and the other is the other way around, to achieve combined effective properties in a single structure made by FGCs. This new inverse design framework provides an opportunity to create shape-morphing structures by utilising modulus-graded composite materials, which can be employed in a variety of applications involving multi-physical environments. Furthermore, this framework underscores the versatility of the approach, enabling precise control over material properties at a local level.

Single exposure passive three-dimensional information reconstruction based on an ordinary imaging system

Existing three-dimensional (3D) imaging technologies have issues such as requiring active illumination, multiple exposures, or coding modulation. We propose a passive single 3D imaging method based on an ordinary imaging system. Using the point spread function of the imaging system to realize the non-coding measurement on the target, the full-focus images and depth information of the 3D target can be extracted from a single two-dimensional (2D) image through the compressed sensing algorithm. Simulation and experiments show that this approach can complete passive 3D imaging based on an ordinary imaging system without any coding operations. This method can achieve millimeter-level vertical resolution under single exposure conditions and has the potential for real-time dynamic 3D imaging. It improves the efficiency of 3D information detection, reduces the complexity of the imaging system, and may be of considerable value to the field of computer vision and other related applications.

3D target detection and spectral classification for single-photon LiDAR data.

3D single-photon LiDAR imaging has an important role in many applications. However, full deployment of this modality will require the analysis of low signal to noise ratio target returns and very high volume of data. This is particularly evident when imaging through obscurants or in high ambient background light conditions. This paper proposes a multiscale approach for 3D surface detection from the photon timing histogram to permit a significant reduction in data volume. The resulting surfaces are background-free and can be used to infer depth and reflectivity information about the target. We demonstrate this by proposing a hierarchical Bayesian model for 3D reconstruction and spectral classification of multispectral single-photon LiDAR data. The reconstruction method promotes spatial correlation between point-cloud estimates and uses a coordinate gradient descent algorithm for parameter estimation. Results on simulated and real data show the benefits of the proposed target detection and reconstruction approaches when compared to state-of-the-art processing algorithms.

Three-Dimensional Tracking of a Target under Angle-Frequency Measurements with Multiple Frequency Lines.

This article considers tracking a constant-velocity underwater target, which emits sound with distinct frequency lines. By analyzing the target's azimuth, elevation and multiple frequency lines, the ownship can estimate the target's position and (constant) velocity. In our paper, this tracking problem is called the 3D Angle-Frequency Target Motion Analysis (AFTMA) problem. We consider the case where some frequency lines disappear and appear occasionally. Instead of tracking every frequency line, this paper proposes to estimate the average emitting frequency by setting the average frequency as the state vector in the filter. As the frequency measurements are averaged, the measurement noise decreases. In the case where we use the average frequency line as our filter state, both the computational load and the root mean square error (RMSE) decrease, compared to the case where we track every frequency line one by one. As far as we know, our manuscript is unique in addressing 3D AFTMA problems, such that an ownship can track an underwater target while measuring the target's sound with multiple frequency lines. The performance of the proposed 3D AFTMA filter is demonstrated utilizing MATLAB simulations.

Feasibility of raised inner strike point equilibria scenario in ITER for detritiation from beryllium co-deposits

In ITER, tritium retention primarily occurs through co-deposition with beryllium. To avoid exceeding the strict tritium inventory limit, efficient tritium recovery techniques are essential. Baking is the ITER baseline for tritium recovery, but its effectiveness in removing tritium from thick beryllium layers is limited. A raised strike point scenario is considered an alternative method for removing tritium from the ITER inner vertical divertor target by heating components via plasma flux. This paper presents SOLPS-ITER code simulations conducted under various conditions, assessing the divertor performance and tritium outgassing of the raised strike point scenario. As the strike point is raised, recycled neutrals are not efficiently baffled by the dome and scrape-off layer, significantly changing the neutral trajectory and ionization source distribution. This improves detachment accessibility but worsens core-edge compatibility compared to the baseline scenario. However, in the partially detached condition, the impact of raising the strike point, perpendicular transport, and q 95 on target heat flux is not significant, as it primarily scales with the input power. Target heat flux is translated to target surface temperature using a simplified heat transfer model that considers the 3D target monoblock geometry and active cooling condition, excluding Be layer thermal properties. For partially detached divertor conditions, the bulk tungsten monoblock surface temperature remains below the baking temperature, which is insufficient for efficient tritium outgassing under the actively cooled ITER divertor condition. However, considering the potential thermal contact resistance between the beryllium and tungsten layers, which may significantly impact temperature distribution, the temperature of the beryllium layer can be raised to a level sufficient for efficient tritium outgassing. Therefore, the raised strike point scenario can be considered as an alternative in-vessel tritium removal technique.

Passive Radar Tracking in Clutter Using Range and Range-Rate Measurements

Passive bistatic radar research is essential for accurate 3D target tracking, especially in the presence of missing or low-quality bearing information. Traditional extended Kalman filter (EKF) methods often introduce bias in such scenarios. To overcome this limitation, we propose employing the unscented Kalman filter (UKF) for handling the nonlinearities in 3D tracking, utilizing range and range-rate measurements. Additionally, we incorporate the probabilistic data association (PDA) algorithm with the UKF to handle cluttered environments. Through extensive simulations, we demonstrate a successful implementation of the UKF-PDA framework, showing that the proposed method effectively reduces bias and significantly advances tracking capabilities in passive bistatic radars.