3D Position Research Articles

A lightweight method of pose estimation for indoor object

Due to the multiple types of objects and the uncertainty of their geometric structures and scales in indoor scenes, the position and pose estimation of point clouds of indoor objects by mobile robots has the problems of domain gap, high learning cost, and high computing cost. In this paper, a lightweight 6D pose estimation method is proposed, which decomposes the pose estimation into a viewpoint and the in-plane rotation around the optical axis of the viewpoint, and the improved PointNet+⁣+ network structure and two lightweight modules are used to construct a codebook, and the 6d pose estimation of the point cloud of the indoor objects is completed by building and querying the codebook. The model was trained on the ShapeNetV2 dataset, and reports the ADD-S metric validation on the YCB-Video and LineMOD datasets, reaching 97.0% and 94.6% respectively. The experiment shows that the model can be trained to estimate the 6d position and pose of the unknown object point cloud with lower computation and storage cost, and the model with fewer parameters and better real-time performance is superior to other high-recision methods.

A Cost-Effective System for Indoor Three-Dimensional Occupant Positioning and Trajectory Reconstruction

Accurate indoor occupancy information extraction plays a crucial role in building energy conservation. Vision-based methods are popularly used for occupancy information extraction because of their high accuracy. However, previous vision-based methods either only provide 2D occupancy information or require expensive equipment. In this paper, we propose a cost-effective indoor occupancy information extraction system that estimates occupant positions and trajectories in 3D using a single RGB camera. The proposed system provides an inverse proportional model to estimate the distance between a human head and the camera according to pixel-heights of human heads, eliminating the dependence on expensive depth sensors. The 3D position coordinates of human heads are calculated based on the above model. The proposed system also associates the 3D position coordinates of human heads with human tracking results by assigning the 3D coordinates of human heads to the corresponding human IDs from a tracking module, obtaining the 3D trajectory of each person. Experimental results demonstrate that the proposed system successfully calculates accurate 3D positions and trajectories of indoor occupants with only one surveillance camera. In conclusion, the proposed system is a low-cost and high-accuracy indoor occupancy information extraction system that has high potential in reducing building energy consumption.

An efficient system for water leak detection and localization based on IoT and lightweight deep learning

Nowadays, Water Distribution Networks (WDNs) efficiency is limited due to leakage issues, which result in large water losses that exceed about one third of the input volume. Considering the problems of growing population and water scarcity especially in Africa and Asia continents, an urgent humanitarian need arisen to efficiently detect and localize pipeline water leaks. To address this need, this paper proposes an end-to-end system for leak detection and localization in water pipelines based on IoT and deep learning. The IoT solution is deployed allowing real time monitoring of pipelines by using non-invasive IoT devices equipped with acoustic sensors which are installed on the pipeline surface. A lightweight one-Dimensional Convolutional Neural Network (1D-CNN) classification model is embedded on these devices to detect leaks. This model uses raw audio signals generated by sensors as input information. Information about the detected leaks is then remotely transmitted using long range and low energy LoRaWAN protocol to the LoRa gateway. The gateway relays messages to a data processing Server. A 1D-CNN regression model has been then created and deployed in this server to directly estimate the 2D position of the leak. As an illustration, An IoT device prototype is designed and used as a case study to test the performance of the proposed system. We demonstrated that our proposal achieves an accuracy of 97 % for detecting leaks and can fix the 2D position of leak with an error of 0.0006 on the X coordinate and 0.0004 on the Y coordinate. Experiments confirm the effectiveness of the proposed solution.

Real-Time Optimal States Estimation with Inertial and Delayed Visual Measurements for Unmanned Aerial Vehicles.

Motion estimation is a major issue in applications of Unmanned Aerial Vehicles (UAVs). This paper proposes an entire solution to solve this issue using information from an Inertial Measurement Unit (IMU) and a monocular camera. The solution includes two steps: visual location and multisensory data fusion. In this paper, attitude information provided by the IMU is used as parameters in Kalman equations, which are different from pure visual location methods. Then, the location of the system is obtained, and it will be utilized as the observation in data fusion. Considering the multiple updating frequencies of sensors and the delay of visual observation, a multi-rate delay-compensated optimal estimator based on the Kalman filter is presented, which could fuse the information and obtain the estimation of 3D positions as well as translational speed. Additionally, the estimator was modified to minimize the computational burden, so that it could run onboard in real time. The performance of the overall solution was assessed using field experiments on a quadrotor system, compared with the estimation results of some other methods as well as the ground truth data. The results illustrate the effectiveness of the proposed method.

Stereo matching method based on high-precision occlusion-recovering and discontinuity-preserving

Stereo matching is the core of machine vision, such as 3D reconstruction, posture perception and driverless technology in the future etc. Currently, there are still great difficulties in improving the matching accuracy of disparity discontinuity and occlusion regions. In this paper, we propose a new high-precision occlusion-recovering and discontinuity-preserving method for stereo matching. We focus on disparity refinement, and present an adaptability-based disparity reconstruction that achieves accurate occlusion-recovering by fusing color aberration, distance, and disparity value. For discontinuity-preserving, disparity-based secondary guided filtering is proposed, then introduce adaptive image enhancement and adaptive Canny edge detection to correct errors in discontinuous regions. The experiments on Middlebury standard dataset show that our algorithm has high overall accuracy, especially in the disparity discontinuity and occlusion regions. This novel method is more direct, efficient, and less sensitive to the accuracy of disparity estimation, and the live-action shooting shows the robustness of our algorithm. The method has the prospect of civil and engineering use, and presents scientific reference for high-precision target identification, target tracking, 3D measurement, etc.

Estimation of 3D anatomically précised hand poses using single shot corrective CNN

A crucial component of human-computer interaction is 3D hand posture assessment. The most recent advancements in computer vision have made estimating 3D hand positions simpler by using deep sensors. The main challenge still stems from unrealistic 3D hand poses because the existing models only use the training dataset to learn the kinematic rules, which is ambiguous, and it is a difficult task to estimate realistic 3D hand poses from datasets because they are not free from anatomical errors. The suggested model in this study is trained using a closed-form expression that encodes the biomechanical rules, thus it does not entirely reliant on the pictures from the annotated dataset. This work also used a Single Shot Detection and Correction convolutional neural network (SSDC-CNN) to handle the issues in imposing anatomically correctness from the architecture level. The ResNetPlus is implemented to improve representation capability with enhanced the efficiency of error back-propagation of the network. The datasets of the Yoga Mudras, like HANDS2017, and MSRA have been used to train and test the future model. As observed from the ground truth the previous hand models have many anatomical errors but, the proposed hand model is anatomically error free hand model compared to previous hand models. By considering the ground truth hand pose, the recommended hand model has shown good accuracy when compared to the state-of-art hand models.

Exploring Spatial-Based Position Encoding for Image Captioning

Image captioning has become a hot topic in artificial intelligence research and sits at the intersection of computer vision and natural language processing. Most recent imaging captioning models have adopted an “encoder + decoder” architecture, in which the encoder is employed generally to extract the visual feature, while the decoder generates the descriptive sentence word by word. However, the visual features need to be flattened into sequence form before being forwarded to the decoder, and this results in the loss of the 2D spatial position information of the image. This limitation is particularly pronounced in the Transformer architecture since it is inherently not position-aware. Therefore, in this paper, we propose a simple coordinate-based spatial position encoding method (CSPE) to remedy this deficiency. CSPE firstly creates the 2D position coordinates for each feature pixel, and then encodes them by row and by column separately via trainable or hard encoding, effectively strengthening the position representation of visual features and enriching the generated description sentences. In addition, in order to reduce the time cost, we also explore a diagonal-based spatial position encoding (DSPE) approach. Compared with CSPE, DSPE is slightly inferior in performance but has a faster calculation speed. Extensive experiments on the MS COCO 2014 dataset demonstrate that CSPE and DSPE can significantly enhance the spatial position representation of visual features. CSPE, in particular, demonstrates BLEU-4 and CIDEr metrics improved by 1.6% and 5.7%, respectively, compared with a baseline model without sequence-based position encoding, and also outperforms current sequence-based position encoding approaches by a significant margin. In addition, the robustness and plug-and-play ability of the proposed method are validated based on a medical captioning generation model.

UAV Trajectory and Multi-User Beamforming Optimization for Clustered Users Against Passive Eavesdropping Attacks With Unknown CSI

This paper tackles the fundamental passive eavesdropping problem in modern wireless communications in which the location and the channel state information (CSI) of the attackers are unknown. In this regard, we propose deploying an unmanned aerial vehicle (UAV) that serves as a mobile aerial relay (AR) to help ground base station (GBS) support a subset of vulnerable users. More precisely, our solution (1) clusters the single-antenna users in two groups to be either served by the GBS directly or via the AR, (2) employs optimal multi-user beamforming to the directly served users, and (3) optimizes the AR's 3D position, its multi-user beamforming matrix and transmit powers by combining closed-form solutions with machine learning techniques. Specifically, we design a plain beamforming and power optimization combined with a deep reinforcement learning (DRL) algorithm for an AR to optimize its trajectory for the security maximization of the served users. Numerical results show that the multi-user multiple input, single output (MU-MISO) system split between a GBS and an AR with optimized transmission parameters without knowledge of the eavesdropping channels achieves high secrecy capacities that scale well with increasing the number of users.

A land vehicle’s INS/GNSS integrated navigation system using left invariant extended kalman filter

Land vehicles need high-precision navigational systems in which multi-sensor integration may be provided. Moreover, land vehicles regularly use Global Navigation Satellite Systems (GNSS) to estimate their position. Unfortunately, several locations, such as tunnels and inside parking garages, where GNSS signals cannot be detected. Several types of research have been conducted to improve positioning information using multi-sensor integration. Then, the vehicle needs another system for finding its location in GNSS-denied conditions, such as Inertial Navigation System (INS). Despite the accuracy of INS in short-time period use, inertial navigation systems (INS) are liable to drifts of their positioning solution due to the inertial sensor errors that are inherent to them; therefore, this problem leads to errors accumulation over time then integration techniques are used to eliminate the resulting errors. Moreover, many filters are used in the process of integration, such as the Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), Particular Filter (PF) and Invariant Extended Kalman Filter (IEKF). Moreover, this work introduces the left-invariant extended Kalman filter (LIEKF) as a navigation filter for a loosely coupled integration to eliminate positioning errors. Furthermore, the LIEKF is based on the symmetry-preserving observer theory, which claims that the estimation error depends on the theory of a Lie group matrix, and the proposed system INS/GPS-based LIEKF converges to constant values, unlike the traditional INS/GPS. Moreover, the proposed system INS/GPS-based LIEKF depends on State-estimate-independent Jacobians, and the LIEKF is more efficient and has better performance due to results such as the 2D position RMS error due to the INS/GPS-based EKF is 19.43m. However, the 2D position RMS error due to the INS/GPS-based LIEKF is 3.32m with 83% improvement. Moreover, the 2D position errors were enhanced using the INS/GPS-based LIEKF system compared to the INS/GPS-based EKF system.

EPT-Net: Edge Perception Transformer for 3D Medical Image Segmentation.

The convolutional neural network has achieved remarkable results in most medical image seg- mentation applications. However, the intrinsic locality of convolution operation has limitations in modeling the long-range dependency. Although the Transformer designed for sequence-to-sequence global prediction was born to solve this problem, it may lead to limited positioning capability due to insufficient low-level detail features. Moreover, low-level features have rich fine-grained information, which greatly impacts edge segmentation decisions of different organs. However, a simple CNN module is difficult to capture the edge information in fine-grained features, and the computational power and memory consumed in processing high-resolution 3D features are costly. This paper proposes an encoder-decoder network that effectively combines edge perception and Transformer structure to segment medical images accurately, called EPT-Net. Under this framework, this paper proposes a Dual Position Transformer to enhance the 3D spatial positioning ability effectively. In addition, as low-level features contain detailed information, we conduct an Edge Weight Guidance module to extract edge information by minimizing the edge information function without adding network parameters. Furthermore, we verified the effectiveness of the proposed method on three datasets, including SegTHOR 2019, Multi-Atlas Labeling Beyond the Cranial Vault and the re-labeled KiTS19 dataset called KiTS19-M by us. The experimental results show that EPT-Net has significantly improved compared with the state-of-the-art medical image segmentation method.

Preface

The 2023 4th International Conference on Internet of Things, Artificial Intelligence and Mechanical Automation (IoTAIMA 2023) was held via hybrid form in Guangzhou, China during July 21st to 23rd, 2023, following the successes of previous events held in Hangzhou (2020, 2021), and Shanghai (2022). This year the IoTAIMA 2023 aimed to bring together researchers, developers, and users in both industry and academia in the world to discuss emerging issues on Internet of Things, artificial intelligence and mechanical automation, for sharing state-of-art results, and for exploring new areas of research and development.The Conference was organized by Peking University-Wuhan Institute for Artificial Intelligence and co-organized by Central South University School of Automation. More than 80 delegates from home and abroad attended the Conference.The IoTAIMA 2023 was featured with four keynote speeches (by Prof. Deyu Qi, Prof. Jinping Ao, Prof. Ming Jiang, Assoc. Prof. Wei Wei, respectively), and several oral and poster presentations, in which a wide range of topics were covered and the most recent significant results were presented. Prof. Ming Jiang from Sun Yat-sen University, China addressed his speech Massive MIMO Aided Multinetwork 3D Positioning. Exploiting a distributed M-MIMO framework, his team proposed to employ a deep belief network (DBN) to analyze the received signal strengths (RSS) generated by a diffraction model. Next, the preliminary DBN estimates were forwarded to a long-short term memory network (LSTMN), where the trajectory information of the targeted UE can be extracted based on much less historical trajectory information than existing solutions. Then, the three-dimension (3D) coordinates of the UE’s positions can be estimated with a back propagation neural network (BPNN) which combines the outputs of DBN and LSTMN. Finally, extensive simulation results were provided to demonstrate the effectiveness of the proposed BPNN scheme.Quantities of excellent papers evaluated based on their originality, technical or research content, correctness, relevance to conference, contributions, and readability were presented in the Conference Proceedings published by Journal of Physics: Conference Series (JPCS). The topics of these papers include Radio Frequency Identification, Micro-Electro-Mechanical Systems, New Sensing Technology, Unmanned System Control Technology, Industrial Robots and Automatic Production line, etc.With the excellent quality of all the presentations, the IoTAIMA 2023 was a great success. We wish to thank the sponsors of the Conference, and particularly the Technical Program Committee. We would also like to extend our gratitude to all the speakers and authors for sharing scientific ideas and presenting new perspectives with us on related topics. Our appreciation also goes to the editors and other staff of JPCS for the assistance it provided for the publication of this paper volume.The Committee of IoTAIMA 2023List of Committee Member is available in this Pdf.

Low-Latency Beaming Display: Implementation of Wearable, 133 $\mu$s Motion-to-Photon Latency Near-eye Display.

This paper presents a low-latency Beaming Display system with 133 μs motion-to-photon (M2P) latency, the latency from head motion to the corresponding image motion. Beaming Display is a recent near-eye display paradigm consisting of a steerable remote projector and a passive wearable headset that aims to overcome the typical trade-offs of OST-HMDs such as weight and computational resources. However, because Beaming Display projects a small image onto a distant moving viewpoint, M2P latency has a significant effect on displacement. To reduce M2P latency, we propose a low-latency Beaming-Display system that can be modularized without expensive high-speed devices. In our system, a 2D position sensor, which is placed coaxially on the projector, detects the light from the IR-LED on the headset and generates a differential signal for tracking. Analog closed-loop control of the steering mirror based on this signal keeps projecting images onto the headset. We implement a proof-of-concept prototype, evaluate the latency and augmented reality experience through a user-perspective camera, and discuss the limitations and improvements of the prototype.

Research on Dynamic Monitoring and Early Warning of the High-Rise Building Machine during the Climbing Stage

High-rise building machines (HBMs) are commonly used for the construction of super-high skyscrapers. Monitoring and early warning are critical to ensure the safety of giant HBMs during dynamic climbing. However, no indicators or control methods directly reflect the dynamic safety state of such large structures. In this study, the key factors influencing the climbing altitude were systematically analyzed and a method for the dynamic monitoring and early warning of HBMs during climbing was proposed. This approach is innovative for monitoring 3D attitude in real time using a string of fiber grating-level sensors mounted on the main bearing surface of the HBM. Three-level early warning indicators and control methods that reflect the safety status of the HBM during dynamic climbing have been established. The method was successfully applied to a 356-m tall high-rise project, the Shenzhen Xinghe Yabao Building. Results demonstrate that the proposed method can monitor and control the safety state of the HBM climbing process more accurately than current methods in real time. In addition, it significantly reduces the impact of factors such as preclimbing differential deformation and climbing attitude recognition on the accuracy of the HBM climbing control. This guarantees the safe management of HBMs and the efficient construction of super high-rise buildings. The results of this study can also be widely applied to safety monitoring of the dynamic operation of giant construction machinery.

FusionCalib: Automatic extrinsic parameters calibration based on road plane reconstruction for roadside integrated radar camera fusion sensors

Calibrating the relative positional posture of the roadside sensor with respect to the road is crucial for target tracking and sensor fusion. However, most previous studies rely on prior information or require additional manual measurements, which increases calibration error and labor costs. This paper proposes an automatic extrinsic parameters calibration method for roadside integrated radar camera fusion sensors for the first time. In particular, we leverage the radar camera fusion framework to automatically reconstruct the road plane. Specifically, to integrate sensing abilities, a stable bidirectional selection association method is adopted to match radar and camera trajectories. Based on the association results, the Extended Kalman Filter (EKF) is adopted to fuse pixel positions and three-dimensional (3D) velocities of vehicles, and then a series of 3D positions of vehicle bottoms are obtained to reconstruct the road plane. Experimental results show that our proposed method achieves automatic and accurate calibration of extrinsic parameters.

A non-contact interactive system for multimodal surgical robots based on LeapMotion and visual tags.

In recent years, the integration of robots in minimally invasive surgery has gained significant traction in clinical practice. However, conventional contact-based human-computer interaction poses the risk of bacterial infection, significantly limiting the role of robots in surgery. To address this limitation, we propose an innovative interaction method rooted in gestures and visual tags, allowing surgeons to control and fine-tune surgical robots without physical contact with the environment. By encoding the six gestures collected using LeapMotion, we can effectively control the surgical robot in a non-contact manner. Moreover, utilizing Aruco technology, we have accurately identified the 3D spatial position of the visual label, and developed 12 fine-tuning operations to refine surgical instruments. To evaluate the applicability of our proposed system in surgery, we designed a relevant experimental setup. In the experiment, we achieved enough precision. These results demonstrate that our system meets the clinical standard, providing doctors with a non-contact and flexible means of interacting with robots during surgery.

Shelf acetabuloplasty may inhibit range of motion: A computer simulation analysis.

Shelf acetabuloplasty has been performed in patients with developmental dysplasia of the hip (DDH); however, the appropriate position of the shelf has not been determined, particularly with respect to avoiding range of motion (ROM) loss. The aim of this study was to investigate the frequency of ROM inhibition and the influence of 3D position of the shelf following actual shelf acetabuloplasty and virtual surgery using computer simulation analysis. Computed tomography data from 15 patients with DDH who underwent shelf acetabuloplasty were collected between August 2019 and February 2022. The three-dimensional models of a hip joint were constructed using Zed Hip®. Maximal internal rotation (MIR) at 45° and 90° flexion was measured in each patient. The frequency and position of ROM inhibition was determined in a real postoperative model virtually. In addition, a second analysis using virtual acetabuloplasty was performed. Upon placing the shelf, three patterns were provided for the following four parameters: height, coronal inclination, center-edge angle (CEA), and anteroposterior position. The predictors for ROM inhibition were analyzed using a logistic regression model. In the actual postoperative model, a limitation of MIR at 90° and 45° of flexion occurred in 60% and 66.7% of patients, respectively. A higher CEA and anterior position are major factors limiting MIR. The analysis of the virtual shelf acetabuloplasty model revealed that anterior position and CEA were significant factors causing ROM inhibition. As for clinical significance, the results of the current study indicated the optimal location of the shelf to avoid iatrogenic impingement after shelf acetabuloplasty.

Scapholunate and lunotriquetral joint dynamic stabilizers and their role in wrist neuromuscular control and proprioception

Recent research interest has grown in exploring the role of muscles, isometric contraction, proprioception, and neuromuscular control in addressing dynamic scapholunate and lunotriquetral joint instability, marking a shift in the understanding of wrist stability. To present a comprehensive review of the carpal ligaments anatomy and wrist biomechanics, with a particular focus on the role of proprioception in dynamic carpal stability and their role in managing scapholunate (SL) and lunotriquetral (LTq) dynamic instabilities. We conducted a systematic search of the literature and review of the most relevant papers published and indexed in pubmed, related to wrist biomechanics, proprioception and its contribution to carpal dynamic stability. The study involved a comprehensive review of neuromuscular mechanisms in dynamic stabilization of the carpus, based on cadaver studies. The 3D position of the scaphoid, triquetrum, and capitate was monitored before and after tendon loading. The extensor carpi ulnaris (ECU) and the flexor carpi radialis (FCR) are identified as the primary pronators of the midcarpal joint. The ECU's pronation effect can potentially strain the scapholunate ligament, while the supinator muscles, the abductor pollicis longus (APL), the extensor carpi radialis longus (ECRL), and the flexor carpi ulnaris (FCU), have a protective role, particularly in cases of scapholunate ligament dysfunctions. The FCR, despite being a pronator of the distal row, has a beneficial effect as it provokes supination of the scaphoid. Comprehending carpal dysfunctions and instabilities hinges on understanding carpal anatomy and normal biomechanics. Proprioception, encompassing joint position sensation and neuromuscular control, is pivotal for stability. Biomechanical research informs tailored muscle strengthening for specific carpal issues. Supinator muscles should be strengthened for SL injuries, and ECU-focused strengthening and proprioceptive training are key for dynamic LTq instabilities. Ongoing research should delve into the intricate relationship between carpal ligaments, muscles, and proprioception to enhance wrist stability.

Heterogeneous Drone Small Cells: Optimal 3D Placement for Downlink Power Efficiency and Rate Satisfaction

In this paper, we delve into the domain of heterogeneous drone-enabled aerial base stations, each equipped with varying transmit powers, serving as downlink wireless providers for ground users. A central challenge lies in strategically selecting and deploying a subset from the available drone base stations (DBSs) to meet the downlink data rate requirements while minimizing the overall power consumption. To tackle this, we formulate an optimization problem to identify the optimal subset of DBSs, ensuring wireless coverage with an acceptable transmission rate in the downlink path. Moreover, we determine their 3D positions for power consumption optimization. Assuming DBSs operate within the same frequency band, we introduce an innovative, computationally efficient beamforming method to mitigate intercell interference in the downlink. We propose a Kalai–Smorodinsky bargaining solution to establish the optimal beamforming strategy, compensating for interference-related impairments. Our simulation results underscore the efficacy of our solution and offer valuable insights into the performance intricacies of heterogeneous drone-based small-cell networks.

Using 3D position sensitivity to reveal response non-uniformities in CdZnTe, TlBr, and CsPbBr3 detectors

Position sensitivity enables the correction of response non-uniformities in room-temperature semiconductor detectors caused by crystal defects and other factors. It can also be used to pinpoint the exact location of crystal defects responsible for the response variations. This work describes a technique for revealing and visualizing the detector regions affecting the charge collection efficiency in CdZnTe (CZT), TlBr, and CsPbBr3 detectors configured as position-sensitive virtual Frisch-grid (VFG) devices. The technique correlates the photopeak events in energy spectra with their spatial distributions inside the detectors using the position information. By selecting the events from narrow energy intervals within a photopeak, we can visualize the detector volumes with particular charge collection efficiencies, which, in turn, correlate with the locations of electrode and crystal defects. We demonstrate this technique in several examples. Columnar structures in the volume plots (position distribution maps) are consistent with signal losses near or at the anode in selected samples of CZT and TlBr. Structures exhibiting a distinct depth dependence are consistent with grain boundaries or other crystal defects.

Volumetric tumor tracking from a single cone-beam X-ray projection image enabled by deep learning

Radiotherapy serves as a pivotal treatment modality for malignant tumors. However, the accuracy of radiotherapy is significantly compromised due to respiratory-induced fluctuations in the size, shape, and position of the tumor. To address this challenge, we introduce a deep learning-anchored, volumetric tumor tracking methodology that employs single-angle X-ray projection images. This process involves aligning the intraoperative two-dimensional (2D) X-ray images with the pre-treatment three-dimensional (3D) planning Computed Tomography (CT) scans, enabling the extraction of the 3D tumor position and segmentation. Prior to therapy, a bespoke patient-specific tumor tracking model is formulated, leveraging a hybrid data augmentation, style correction, and registration network to create a mapping from single-angle 2D X-ray images to the corresponding 3D tumors. During the treatment phase, real-time X-ray images are fed into the trained model, producing the respective 3D tumor positioning. Rigorous validation conducted on actual patient lung data and lung phantoms attests to the high localization precision of our method at lowered radiation doses, thus heralding promising strides towards enhancing the precision of radiotherapy.