Fast 3D Research Articles

Diffuser-based fiber endoscopy for single-shot 3D fluorescence imaging

Minimally invasive endoscopy using coherent fiber bundles shows great potential for numerous applications in biomedical imaging. With a diffuser on the distal side of the fiber bundle and computational image recovery, single-shot 3D imaging is possible by encoding the image volume into 2D speckle patterns. In comparison to equivalent lens systems, a higher space-bandwidth product can be achieved. However, decoding the image with iterative algorithms is time-consuming. Thus, we propose utilizing a neural network for fast 2D and 3D image reconstruction at video rate. In this work, single-shot 3D fluorescence imaging with an ultra-thin endoscope is demonstrated, enabling applications like calcium imaging for in vivo brain diagnostics at cellular resolution.

3D Visualization of Pulmonary Vessel Based on Low-Cost Segmentation and Fast Reconstruction

Real-time visual-aided navigation and path strategy for pneumonoconiosis and efficient 3D visualization of pulmonary vessels are of great research and clinical significance in the treatment of lung diseases. The complex structure of lung tissue limits the application of deep learning in pulmonary vascular visualization due to the lack of vascular labeling datasets. Also, the existing methods have large computational complexity and are low efficiency. This study proposes a method for high-quality 3D visualization of pulmonary vessels based on low-cost segmentation and fast reconstruction, consisting of three steps: 1) Pulmonary vessel feature extraction from lung CT images using self-supervised learning, 2) Segmentation of pulmonary sparse vessels in lung CT images using self-supervised transfer learning, and 3) 3D reconstruction of pulmonary vessels based on segmentation results of step (2) using interpolation. The accuracy of pulmonary vascular contour segmentation was improved from 91.31% using the sparse coding to 98.65% using our proposed method (27,270 test sample points); the classifier evaluation accuracy was improved from 95.33% to 98.26%, and the average running time of the model with the test set data was 44 ms per slice. the segmentation results can automatically generate a complete vascular tree model with an average time of 10.8s ± 1 1.6s. The results demonstrate that the proposed method provides fast and accurate 3D visualization of pulmonary vessels, and is promising for more precise and reliable information for pneumonoconiosis patients.

Fast and High Accuracy 3D Point Cloud Registration for Automatic Reconstruction From Laser Scanning Data

Point cloud registration from laser scanning data is a technique to establish the mapping relationship between source and target point clouds, which has been widely used in automatic 3D reconstruction, pose estimation, localization, and navigation. While algorithms like Super4PCS and MSSF-4PCS can achieve registration without initial poses, they are relatively slow, less accurate, and require iterations. To address these issues, we propose a 3D point cloud registration algorithm based on interval segmentation and multi-dimensional feature. Firstly, the source and target point clouds are segmented internally and the point cloud curvature is designed to narrow down the search range for the registration between the segmented point clouds. Secondly, the corresponding four-point sets in the segmented areas of the source and target point clouds are determined using affine invariance constraints. Finally, a multi-dimensional feature vector based on curvature features and fast point feature histogram is established to determine the unique corresponding four-point set pairs, and the rigid body transformation matrix is solved accordingly. Our algorithm is tested on publicly available 3D point cloud data models Bunny, Dino, Dragon, and Horse from Stanford University. Results showed that our algorithm improved registration accuracy by 24.39% and registration efficiency by 46.21% compared to the MSSF-4PCS point cloud registration algorithm. Multiple sets of experimental results confirmed the effectiveness of our algorithm. The proposed 3D point cloud registration is proved to be fast with high accuracy, which can be utilized for automatic segmentation, reconstruction, and modelling from Laser Scanning Data.

TomocuPy - efficient GPU-based tomographic reconstruction with asynchronous data processing.

Fast 3D data analysis and steering of a tomographic experiment by changing environmental conditions or acquisition parameters require fast, close to real-time, 3D reconstruction of large data volumes. Here a performance-optimized TomocuPy package is presented as a GPU alternative to the commonly used central processing unit (CPU) based TomoPy package for tomographic reconstruction. TomocuPy utilizes modern hardware capabilities to organize a 3D asynchronous reconstruction involving parallel read/write operations with storage drives, CPU-GPU data transfers, and GPU computations. In the asynchronous reconstruction, all the operations are timely overlapped to almost fully hide all data management time. Since most cameras work with less than 16-bit digital output, the memory usage and processing speed are furthermore optimized by using 16-bit floating-point arithmetic. As a result, 3D reconstruction with TomocuPy became 20-30 times faster than its multi-threaded CPU equivalent. Full reconstruction (including read/write operations and methods initialization) of a 20483 tomographic volume takes less than 7 s on a single Nvidia Tesla A100 and PCIe 4.0 NVMe SSD, and scales almost linearly increasing the data size. To simplify operation at synchrotron beamlines, TomocuPy provides an easy-to-use command-line interface. Efficacy of the package was demonstrated during a tomographic experiment on gas-hydrate formation in porous samples, where a steering option was implemented as a lens-changing mechanism for zooming to regions of interest.

Towards Single Camera Human 3D-Kinematics.

Markerless estimation of 3D Kinematics has the great potential to clinically diagnose and monitor movement disorders without referrals to expensive motion capture labs; however, current approaches are limited by performing multiple de-coupled steps to estimate the kinematics of a person from videos. Most current techniques work in a multi-step approach by first detecting the pose of the body and then fitting a musculoskeletal model to the data for accurate kinematic estimation. Errors in training data of the pose detection algorithms, model scaling, as well the requirement of multiple cameras limit the use of these techniques in a clinical setting. Our goal is to pave the way toward fast, easily applicable and accurate 3D kinematic estimation. To this end, we propose a novel approach for direct 3D human kinematic estimation D3KE from videos using deep neural networks. Our experiments demonstrate that the proposed end-to-end training is robust and outperforms 2D and 3D markerless motion capture based kinematic estimation pipelines in terms of joint angles error by a large margin (35% from 5.44 to 3.54 degrees). We show that D3KE is superior to the multi-step approach and can run at video framerate speeds. This technology shows the potential for clinical analysis from mobile devices in the future.

Heterogeneous stacking of reduced graphene oxide on ZnO nanowires for NO2 gas sensors with dramatically improved response and high sensitivity

Graphene or two-dimensional materials have been intensively studied as a new generation of gas sensing materials due to their large specific surface area and high mobility. However, fabrication processes for oxide and 2D materials lead to non-uniform structures of flakes of graphene or its derivatives and oxide nanowire, are randomly suspended to devices, resulting in poor and unrepeatable sensing performances. Here, we report on the heterogeneous stacked interface of reduced graphene oxide (rGO) on the surface of ZnO nanowires and their demonstration as a NO2 gas sensor. Compared to the conventional surface decoration using noble metals such as Au, Ag, and Pd, the present sensor shows excellent sensing performances including 22 times faster response behavior. Moreover, this interface-based rGO-ZnO gas sensor showed outperforming sensitivity and recovery time to reported 2D and 2D/oxide based gas sensor. The active sites of rGO are more favorable for chemisorption of oxygen molecules due to functional groups on rGO surfaces. Moreover, the gas-sensing mechanism is firstly elucidated by the finite-difference time-domain (FDTD) simulation, confirming that mono-to-few layers of rGO on ZnO act a role of bridge, facilitating the migration of electrons from ZnO to NO2, leading to higher increment of depletion region and corresponding sensor response. Our approaches may offer the new opportunities and strategies for highly sensitive and fast recoverable 2D materials/oxide hybrid sensors.

Toward fast 3D human activity recognition: A refined feature based on minimum joint freedom model (Mint)

Human activity recognition (HAR) is crucial for human-centric smart manufacturing, especially the management of human involved production activities. Recently, 3D HAR based on skeleton data has emerged as an outstanding approach for good performance on both speed and accuracy. However, there are still flaws like insufficiency or redundancy in design of features. To address this gap, a refined skeleton-based feature for fast and accurate 3D HAR is proposed in this work, Specifically, a minimum and lossless skeleton feature named Mint is proposed based on minimum joint freedom model, with a reduction of around 52% skeleton data dimensions to classic coordinate-based feature. For demonstration, several comparison experiments are implemented based on the presented feature and classic coordinate-based feature. Results show that the utilization of proposed feature can improve the efficiency of 3D HAR around 29% with equivalent accuracy. It is expected this work can contribute a lot to the application of HAR in human-centric smart manufacturing systems and real-time human activity digital twin (HADT) modelling and interaction.

Monitoring neonatal brain hemorrhage progression by photoacoustic tomography.

Neonatal brain hemorrhage (NBH) is the most common neurological disorder in neonates and its clinical interventions are very limited. Understanding the pathology of NBH by non-invasive in-vivo characterization of standardized animal models is essential for developing potential treatments. Currently, there is no suitable tool to provide non-invasive, non-ionizing dynamic imaging of neonatal mouse models with high resolution, high contrast, and deep imaging depth. In this study, we implemented a fast 3D photoacoustic tomography (PAT) system suitable for imaging neonatal mouse brains with good image quality and demonstrated its feasibility in non-invasive monitoring of the dynamic process of NBH in the whole neonatal mouse brain. The results present a high resolution and sensitivity for NBH detection. Both morphological and hemodynamic changes of the hematoma were accurately obtained. Our results demonstrated the potential of PAT as a powerful tool for the preclinical study of neonatal brain hemorrhage.

Advances in machine learning applications for cardiovascular 4D flow MRI.

Four-dimensional flow magnetic resonance imaging (MRI) has evolved as a non-invasive imaging technique to visualize and quantify blood flow in the heart and vessels. Hemodynamic parameters derived from 4D flow MRI, such as net flow and peak velocities, but also kinetic energy, turbulent kinetic energy, viscous energy loss, and wall shear stress have shown to be of diagnostic relevance for cardiovascular diseases. 4D flow MRI, however, has several limitations. Its long acquisition times and its limited spatio-temporal resolutions lead to inaccuracies in velocity measurements in small and low-flow vessels and near the vessel wall. Additionally, 4D flow MRI requires long post-processing times, since inaccuracies due to the measurement process need to be corrected for and parameter quantification requires 2D and 3D contour drawing. Several machine learning (ML) techniques have been proposed to overcome these limitations. Existing scan acceleration methods have been extended using ML for image reconstruction and ML based super-resolution methods have been used to assimilate high-resolution computational fluid dynamic simulations and 4D flow MRI, which leads to more realistic velocity results. ML efforts have also focused on the automation of other post-processing steps, by learning phase corrections and anti-aliasing. To automate contour drawing and 3D segmentation, networks such as the U-Net have been widely applied. This review summarizes the latest ML advances in 4D flow MRI with a focus on technical aspects and applications. It is divided into the current status of fast and accurate 4D flow MRI data generation, ML based post-processing tools for phase correction and vessel delineation and the statistical evaluation of blood flow.

Fast absolute 3D CGO-based electrical impedance tomography on experimental tank data

Objective. To present the first 3D CGO-based absolute EIT reconstructions from experimental tank data. Approach. CGO-based methods for absolute EIT imaging are compared to traditional TV regularized non-linear least squares reconstruction methods. Additional robustness testing is performed by considering incorrect modeling of domain shape. Main Results. The CGO-based methods are fast, and show strong robustness to incorrect domain modeling comparable to classic difference EIT imaging and fewer boundary artefacts than the TV regularized non-linear least squares reference reconstructions. Significance. This work is the first to demonstrate fully 3D CGO-based absolute EIT reconstruction on experimental data and also compares to TV-regularized absolute reconstruction. The speed (1–5 s) and quality of the reconstructions is encouraging for future work in absolute EIT.

Fast 3D computations of compressible flow discharge in buildings and complex networks

This paper presents under-resolved simulations of compressible flows in complex structures like buildings. The mean pressure field is the primary focus as it is of high importance in many cases, such as the overpressure generated by the detonation of an explosive charge. In such high-risk situations, the mean pressure in the different building rooms has to be determined as quickly as possible, in order to help first responders evaluating the risk level of entering the building right after an explosion for instance. Conventional computational fluid dynamics methods are obviously able to deliver those requested pressure fields. However, they also require a long and tedious meshing process due to the presence of small openings in the computational domain (doors, windows or stairwells). In turn, these small openings need spatial and temporal resolution having significant consequences on computation time. But when dealing with pressing situations, a long pre-processing is not acceptable. The present paper proposes a two-step solution method. First, a very rough mesh is constructed which does not consider small openings. The mesh is constructed in an easy and particular way such that simplifications can be made when computing the fluid flux through these openings. Second, a specific Riemann solver dealing with geometric discontinuities is developed to estimate the fluid flow. The Riemann solver takes advantage of the simplifications provided by the meshing method, and accounts for both unchoked and choked flows through openings. The overall method then consists of both a specific meshing process that requires very little pre-processing, which considerably reduces the time needed to produce a complete simulation, and a Riemann solver. The proposed method is named MUZO in reference to its “MUlti-ZOne” flow solver. The MUZO solver is validated against resolved computations and provides accurate simulated mean pressure fields within a realistic building in a few seconds.

Translational rapid ultraviolet-excited sectioning tomography for whole-organ multicolor imaging with real-time molecular staining.

Rapid multicolor three-dimensional (3D) imaging for centimeter-scale specimens with subcellular resolution remains a challenging but captivating scientific pursuit. Here, we present a fast, cost-effective, and robust multicolor whole-organ 3D imaging method assisted with ultraviolet (UV) surface excitation and vibratomy-assisted sectioning, termed translational rapid ultraviolet-excited sectioning tomography (TRUST). With an inexpensive UV light-emitting diode (UV-LED) and a color camera, TRUST achieves widefield exogenous molecular-specific fluorescence and endogenous content-rich autofluorescence imaging simultaneously while preserving low system complexity and system cost. Formalin-fixed specimens are stained layer by layer along with serial mechanical sectioning to achieve automated 3D imaging with high staining uniformity and time efficiency. 3D models of all vital organs in wild-type C57BL/6 mice with the 3D structure of their internal components (e.g., vessel network, glomeruli, and nerve tracts) can be reconstructed after imaging with TRUST to demonstrate its fast, robust, and high-content multicolor 3D imaging capability. Moreover, its potential for developmental biology has also been validated by imaging entire mouse embryos (~2 days for the embryo at the embryonic day of 15). TRUST offers a fast and cost-effective approach for high-resolution whole-organ multicolor 3D imaging while relieving researchers from the heavy sample preparation workload.

CAN3D: Fast 3D medical image segmentation via compact context aggregation.

Direct automatic segmentation of objects in 3D medical imaging, such as magnetic resonance (MR) imaging, is challenging as it often involves accurately identifying multiple individual structures with complex geometries within a large volume under investigation. Most deep learning approaches address these challenges by enhancing their learning capability through a substantial increase in trainable parameters within their models. An increased model complexity will incur high computational costs and large memory requirements unsuitable for real-time implementation on standard clinical workstations, as clinical imaging systems typically have low-end computer hardware with limited memory and CPU resources only. This paper presents a compact convolutional neural network (CAN3D) designed specifically for clinical workstations and allows the segmentation of large 3D Magnetic Resonance (MR) images in real-time. The proposed CAN3D has a shallow memory footprint to reduce the number of model parameters and computer memory required for state-of-the-art performance and maintain data integrity by directly processing large full-size 3D image input volumes with no patches required. The proposed architecture significantly reduces computational costs, especially for inference using the CPU. We also develop a novel loss function with extra shape constraints to improve segmentation accuracy for imbalanced classes in 3D MR images. Compared to state-of-the-art approaches (U-Net3D, improved U-Net3D and V-Net), CAN3D reduced the number of parameters up to two orders of magnitude and achieved much faster inference, up to 5 times when predicting with a standard commercial CPU (instead of GPU). For the open-access OAI-ZIB knee MR dataset, in comparison with manual segmentation, CAN3D achieved Dice coefficient values of (mean = 0.87±0.02 and 0.85±0.04) with mean surface distance errors (mean = 0.36±0.32mm and 0.29±0.10mm) for imbalanced classes such as (femoral and tibial) cartilage volumes respectively when training volume-wise under only 12G video memory. Similarly, CAN3D demonstrated high accuracy and efficiency on a pelvis 3D MR imaging dataset for prostate cancer consisting of 211 examinations with expert manual semantic labels (bladder, body, bone, rectum, prostate) now released publicly for scientific use as part of this work.

Effect of temperature on evaporation dynamics of sheep's blood droplets and topographic analysis of induced patterns

To characterize various induced phenomena and the blood of healthy sheep using several parameters, the evaporation dynamics of 72 drops of sheep blood evaporated at several temperatures: 23, 37, 60, and 90 °C on glass hydrophilic substrates were studied. This allows the prediction of the sheep blood pattern, knowing the surface temperature and vice versa. To determine the variation in the Marangoni number between the center and the triple line, an infrared thermography method was used to measure the temperature variation along the surface of the drop. Simultaneously, a high-performance camera was used to measure the variation in the height of the drop during the evaporation using a superior algorithm software for image analysis, drop shape analyzer, under controlled conditions (Humidity = 40%, Tatm = 23 °C). The study of the evaporation dynamics and pattern formation shows the effect of temperature on the flow circulation inside the drop, resulting in the final deposit. The results showed two categories corresponding to two different evaporation phenomena induced by the thermal Marangoni effect. Furthermore, to transform the induced pattern of sheep blood evaporation into a 3D image, a topographic study was performed using a highly accurate, fast, and flexible optical 3D measurement system. The topographic parameters were subsequently extracted from these 3D images. The statistical study showed a good correlation between the topographic parameters and the surface temperature, and a significant difference between each temperature group for each parameter.

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients.

The 3D reconstruction of an accurate face model is essential for delivering reliable feedback for clinical decision support. Medical imaging and specific depth sensors are accurate but not suitable for an easy-to-use and portable tool. The recent development of deep learning (DL) models opens new challenges for 3D shape reconstruction from a single image. However, the 3D face shape reconstruction of facial palsy patients is still a challenge, and this has not been investigated. The contribution of the present study is to apply these state-of-the-art methods to reconstruct the 3D face shape models of facial palsy patients in natural and mimic postures from one single image. Three different methods (3D Basel Morphable model and two 3D Deep Pre-trained models) were applied to the dataset of two healthy subjects and two facial palsy patients. The reconstructed outcomes were compared to the 3D shapes reconstructed using Kinect-driven and MRI-based information. As a result, the best mean error of the reconstructed face according to the Kinect-driven reconstructed shape is 1.5±1.1 mm. The best error range is 1.9±1.4 mm when compared to the MRI-based shapes. Before using the procedure to reconstruct the 3D faces of patients with facial palsy or other facial disorders, several ideas for increasing the accuracy of the reconstruction can be discussed based on the results. This present study opens new avenues for the fast reconstruction of the 3D face shapes of facial palsy patients from a single image. As perspectives, the best DL method will be implemented into our computer-aided decision support system for facial disorders.

Stochastic multi-fidelity joint hydrogeophysical inversion of consistent geological models

In Quaternary deposits, the characterization of subsurface heterogeneity and its associated uncertainty is critical when dealing with groundwater resource management. The combination of different data types through joint inversion has proven to be an effective way to reduce final model uncertainty. Moreover, it allows the final model to be in agreement with a wider spectrum of data available on site. However, integrating them stochastically through an inversion is very time-consuming and resource expensive, due to the important number of physical simulations needed. The use of multi-fidelity models, by combining low-fidelity inexpensive and less accurate models with high-fidelity expensive and accurate models, allows one to reduce the time needed for inversion to converge. This multiscale logic can be applied for the generation of Quaternary models. Most Quaternary sedimentological models can be considered as geological units (large scale), populated with facies (medium scale), and finally completed by physical parameters (small scale). In this paper, both approaches are combined. A simple and fast time-domain EM 1D geophysical direct problem is used to first constrain a simplified stochastic geologically consistent model, where each stratigraphic unit is considered homogeneous in terms of facies and parameters. The ensemble smoother with multiple data assimilation (ES-MDA) algorithm allows generating an ensemble of plausible subsurface realizations. Fast identification of the large-scale structures is the main point of this step. Once plausible unit models are generated, high-fidelity transient groundwater flow models are incorporated. The low-fidelity models are populated stochastically with heterogeneous facies and their associated parameter distribution. ES-MDA is also used for this task by directly inferring the property values (hydraulic conductivity and resistivity) from the generated model. To preserve consistency, geophysical and hydrogeological data are inverted jointly. This workflow ensures that the models are geologically consistent and are therefore less subject to artifacts due to localized poor-quality data. It is able to robustly estimate the associated uncertainty with the final model. Finally, due to the simplification of both the direct problem and the geology during the low-fidelity part of the inversion, it greatly reduces the time required to converge to an ensemble of complex models while preserving consistency.

Fast spatial structure constrained 3D prestack inversion based on the Sylvester equation

Structurally constrained inversion (SCI) is a multitrace processing technology that can effectively reduce the complexity of prestack seismic inversion caused by complex geology such as faults and steeply dipping structures. Nevertheless, several factors limit its wide application in 3D prestack inversion, the most significant of which is the high computational cost. In addition, the existing SCI is mostly applied in 2D profiles, which can only consider structural features in one direction, making it challenging to ensure spatial continuity. We have developed a fast spatial structure constrained inversion for 3D prestack seismic data. First, based on the conventional SCI, we develop a new way to describe the spatial structural characteristics of the stratum by estimating the dip angles along two perpendicular directions. Second, we introduce the spatial structure constraints through the Hadamard product operator and construct a computationally well-behaved objective function with spatial structural constrained. Using our matrix operators, which are symbols representing the multiplication of 2D and 3D matrices, the minimization of the objective function can be simplified to solve a specific Sylvester equation. Moreover, we justify the rationality of our operator symbols and derive a fast algorithm for solving 3D problems with the Sylvester equation. Finally, the stability and computational efficiency of our method are compared with the 2D SCI algorithm on the 3D overthrust model and field prestack seismic data.

CapSense: A Real-Time Capacitive Sensor Simulation Framework for Physical Human-Robot Interaction

This letter presents CapSense, a real-time open-source capacitive sensor simulation framework for robotic applications. CapSense provides raw data of capacitive proximity sensors based on a fast and efficient 3D finite-element method (FEM) implementation. The proposed framework is interfaced to off-the-shelf robot and physics simulation environments to couple dynamic interaction of the environment with an electrostatic solver for capacitance computation in real-time. The FEM method proposed in this letter relies on a static tetrahedral mesh of the sensor surrounding without a-posteriori re-meshing and achieves high update rates by an adaptive update step. CapSense is flexible due to various configuration parameters (i.e. number, size, shape and location of electrodes) and serves as a platform for investigation of capacitive sensors in robotic applications. By using the proposed framework, researchers can simulate capacitive sensors in different scenarios and investigate these sensors and their configuration prior to installation and fabrication of real hardware. The proposed framework opens new research opportunities via sim-to-real transfer of capacitive sensing. The simulation approach is validated by comparing real-world results of different scenarios with simulation results. In order to showcase the benefits of CapSense in physical Human-Robot Interaction (pHRI), the framework is evaluated in a robotic healthcare scenario.

Fast and Robust Multi-Person 3D Pose Estimation and Tracking From Multiple Views.

This paper addresses the problem of reconstructing 3D poses of multiple people from a few calibrated camera views. The main challenge of this problem is to find the cross-view correspondences among noisy and incomplete 2D pose predictions. Most previous methods address this challenge by directly reasoning in 3D using a pictorial structure model, which is inefficient due to the huge state space. We propose a fast and robust approach to solve this problem. Our key idea is to use a multi-way matching algorithm to cluster the detected 2D poses in all views. Each resulting cluster encodes 2D poses of the same person across different views and consistent correspondences across the keypoints, from which the 3D pose of each person can be effectively inferred. The proposed convex optimization based multi-way matching algorithm is efficient and robust against missing and false detections, without knowing the number of people in the scene. Moreover, we propose to combine geometric and appearance cues for cross-view matching. Finally, an efficient tracking method is proposed to track the detected 3D poses across the multi-view video. The proposed approach achieves the state-of-the-art performance on the Campus and Shelf datasets, while being efficient for real-time applications.

Predicting 4D liver MRI for MR-guided interventions.

Organ motion poses an unresolved challenge in image-guided interventions like radiation therapy, biopsies or tumor ablation. In the pursuit of solving this problem, the research field of time-resolved volumetric magnetic resonance imaging (4D MRI) has evolved. However, current techniques are unsuitable for most interventional settings because they lack sufficient temporal and/or spatial resolution or have long acquisition times. In this work, we propose a novel approach for real-time, high-resolution 4D MRI with large fields of view for MR-guided interventions. To this end, we propose a network-agnostic, end-to-end trainable, deep learning formulation that enables the prediction of a 4D liver MRI with respiratory states from a live 2D navigator MRI. Our method can be used in two ways: First, it can reconstruct high quality fast (near real-time) 4D MRI with high resolution (209×128×128 matrix size with isotropic 1.8mm voxel size and 0.6s/volume) given a dynamic interventional 2D navigator slice for guidance during an intervention. Second, it can be used for retrospective 4D reconstruction with a temporal resolution of below 0.2s/volume for motion analysis and use in radiation therapy. We report a mean target registration error (TRE) of 1.19±0.74mm, which is below voxel size. We compare our results with a state-of-the-art retrospective 4D MRI reconstruction. Visual evaluation shows comparable quality. We compare different network architectures within our formulation. We show that small training sizes with short acquisition times down to 2 min can already achieve promising results and 24 min are sufficient for high quality results. Because our method can be readily combined with earlier time reducing methods, acquisition time can be further decreased while also limiting quality loss. We show that an end-to-end, deep learning formulation is highly promising for 4D MRI reconstruction.