Real-time 3D Research Articles

One-shot domain adaptive real-time 3D obstacle detection in farmland based on semantic-geometry-intensity fusion strategy

By introducing deep learning, LiDAR-based solutions have achieved impressive accuracy in 3D obstacle detection. However, gathering and labeling sufficient samples is the precondition for the effectiveness of existing solutions. This precondition is difficult to satisfy in actual farmland due to the scarcity of distinctive obstacle samples as well as the time-consuming and specialized labeling process. In practice, detection models trained on specific datasets may fail to generalize well to real farmland, as they lack adaptability to different categories and scenes. To address this limitation, this paper proposes a novel one-shot domain adaptive real-time 3D obstacle detection method based on semantic-geometry-intensity fusion strategy. By introducing the concept of one-shot domain adaptation, the proposed method enables fine-grained 3D obstacle detection with just one sample per category. Specifically, a semantic-geometry-intensity space generator is designed to bridge the category gap between training and test samples. The integration of semantic-geometry-intensity space-based classifier and centerpoint-based anchor-free locator is designed to keep a decent balance between accuracy and efficiency. The switching between object sample enhancer and fusion point cloud generator is designed to handle distribution differences of both points and categories. The obstacle detection system designed based on the proposed method has been tested in real farmland, achieving an overall F1 score of 89.54% and a Frame Rate of 21.32 Frames Per Second (FPS). These experimental results demonstrate the high accuracy and efficiency of the proposed method in performing obstacle detection.

Real Time Volumetric MRI for MR-Guided 3D Motion Tracking via Sparse Prior-Augmented Neural Representation Learning

To reconstruct volumetric MRI from orthogonal cine acquisition aided by sparse priors of 2 static 3D MRI through implicit neural representation (NeRP) learning, with the goal of eliminating large-scale training datasets for data-driven sparse MRI reconstruction and supporting clinical workflow of real time 3D motion tracking during MR-guided radiotherapy. A multi-layer perceptron network was trained to learn the NeRP of a patient-specific MRI dataset, where the network takes 4D data coordinates of voxel locations and motion states as inputs and outputs corresponding voxel intensities. By first learning the NeRP of 2 static 3D MRI with different breathing motion states, prior knowledge of patient breathing motion was embedded into network weights through optimization. The prior knowledge was then augmented from 2 to 31 motion states by querying the optimized network at interpolated/extrapolated motion state coordinates. Starting from the prior-augmented network as an initialization point, the network was further trained using sparse samples of 2 orthogonal cine slices. The final volumetric reconstruction was obtained by querying the trained network at desired 3D spatial locations. We evaluated the proposed method using 5-minute volumetric MRI time series with 340 ms temporal resolution collected from 7 liver carcinoma patients. The time series was acquired using golden-angle radial MRI sequence and reconstructed through retrospective sorting. Two MRI with inhale and exhale states respectively were selected from the first 30 sec of the time series for prior embedding and augmentation. The remaining 4.5-min time series was used for volumetric reconstruction evaluation, where we retrospectively subsampled each MRI to 2 orthogonal slices and compared network-reconstructed images to ground truth images in terms of image quality and the capability of supporting 3D target motion tracking. Across the 7 patients evaluated, the peak signal to noise ratio between model reconstruction and ground truth was 54.66 ± 6.16 dB and the structural similarity index measure was 0.99 ± 0.01. Gross tumor volume (GTV) contours estimated by deforming a reference state MRI to model-reconstructed and ground truth MRI showed good consistency. The 95-percentile Hausdorff distance between GTV contours was 1.89 ± 1.13 mm, which is less than the voxel dimension. The mean GTV centroid position difference between ground truth and model estimation was less than 1 mm in all 3 orthogonal directions. Volumetric MRI from orthogonal cine acquisition with sparse priors is feasible by modeling prior knowledge through implicit neural representation learning. The model-reconstructed images showed sufficient accuracy in supporting 3D motion tracking of abdominal targets. By eliminating the need for large scale training datasets, the method promises to enable clinical implementation of 3D motion tracking for precision radiation therapy.

Dosimetric Impact of Intrafraction Prostate Motion and Interfraction Anatomical Changes in Dose-Escalated Linac-Based SBRT

The aim of this study was to investigate the impact of intrafraction prostate motion and interfraction anatomical changes on dose metrics and the effect of beam gating and motion correction in dose-escalated Linac-based SBRT. Thirteen patients (56 fractions) with organ-confined prostate cancer underwent dose-escalated FFF-VMAT SBRT. Accurate patient setup was ensured by CBCT, and real-time 3D prostate motion data were obtained using a novel electromagnetic tracking device. Treatment was interrupted when the signals exceeded a 2 mm threshold in any of the three spatial directions and couch position was corrected unless the offset was transient. Prostate trajectories with and without beam gating and motion correction events were reconstructed and analyzed. Rectum and bladder volumes on each daily CBCT were recorded and compared with volumes at simulation. The prostate motion observed for each fraction was incorporated into the patient original treatment plan with an isocenter shift method. Actually, delivered treatments were then obtained by recalculating the reconstructed motion-encoded plan on deformed CTs reflecting the patient CBCT-anatomy of the day. Non-gated treatments were also simulated using the prostate motion data assuming that no treatment interruptions have occurred. Treatment interruptions because of target motion trespassing the predefined threshold occurred in 25 fractions (45%). Rectum and bladder volume changes were relevant in most patients. The mean relative dose differences between actually delivered and planned treatments were -3.0% [-18.5 - 2.8] for CTV D99% and -2.6% [-17.8 - 1.0] for PTV D95% over all 56 fractions. However, the median cumulative CTV coverage with 93% of the prescribed dose turned out to be satisfactory. Urethra sparing was slightly degraded and a mean reduction of rectum and bladder dose was seen in all but two dose metrics: the maximum dose to rectum mucosa and the bladder D40%. Nevertheless, only 2 major deviations in rectum mucosa D0.035cc were observed at the end of the treatment. The greatest contribution to target missing and OARs doses came from the interfraction anatomical variations, while intrafraction prostate motion marginally contributed in gated treatments. In non-gated treatments, further deteriorations by 2.4 - 2.8% in minimum target coverage metrics and by 3.1 - 11.6% in bladder dose parameters would have occurred on average. The implemented motion management strategy and the strict patient preparation regimen, along with current PTV margins, robustness of original treatment plans, and fast FFF beam delivery, ensured no significant degradations of dose metrics due to both intrafraction motion and interfraction anatomical changes. Non-gated treatments would have more widely deteriorated the dosimetry of some individual fractions. Thus, continuous monitoring, beam gating and motion correction are recommended to safely deliver dose-escalated prostate SBRT.

Hybrid-Supervised Deep Learning for Proton-Acoustic Reconstruction for 3D In Vivo Proton Dose Verification

Proton-acoustic (PA) image has shown great potential to provide real-time 3D dose verification of proton therapy. However, the PA image quality suffers from severe limited view artifacts, which significantly impairs its accuracy for dose verification. In this study, we developed a hybrid-supervised deep learning method for PA reconstruction to address the limited-view issues. Our method consists of two stages. In the first stage, a transformer-based network was proposed to reconstruct initial pressure maps from protoacoustic signals. The network was first trained using supervision by the iteratively reconstructed pressure map and then fine-tuned using transfer learning and self-supervision based on the data fidelity constraint. In the second stage, the PA image was further enhanced by a 3D U-net. The final PA images were converted to dose maps using conversion coefficients derived from CT images. Data from 126 prostate cancer patients treated by proton therapy were collected under an IRB protocol and were split into 86 and 40 patients for model training and testing, respectively. Data of each patient contains the planning CT scan, the corresponding clinical treatment plan, and the dose map calculated by commercial software. The radiofrequency signals were generated by performing proton acoustic simulation based on CT images and the ground truth pressure map derived from the treatment plan. An ultrasound detector matrix with 64 × 64 size and 500kHz central frequency was simulated under the perineum to acquire the signals in the prostate area. In the testing results, the method's accuracy was evaluated using Root-mean-squared-error (RMSE) and structural-similarity-index-measure (SSIM) between the reconstructed and ground truth pressure map and dose distribution. Testing results showed that the reconstructed pressure map achieved an average RMSE/SSIM of 0.0292/0.96, demonstrating excellent 3D information with details. Dose maps derived from the pressure map achieved an average RMSE/SSIM of 0.018/0.99 with a gamma index of 94.7% and 95.7% for 1%/3 mm and 1%/5 mm criteria compared to the ground truth dose maps. The reconstruction time was 6s, which can be further reduced using GPU. Our study achieves start-of-the-art performance in the challenging task of direct reconstruction from limited-view radiofrequency signals, demonstrating the great promise of PA imaging as a highly efficient and accurate tool for in-vivo 3D proton dose verification. Such high-precision 3D online dose verification can substantially reduce the range uncertainties of proton therapy to significantly improve its precision and outcomes.

Real-Time 3D Liver Tumor Localization via Combined Optical Surface Imaging and an X-Ray Projection from Arbitrary Imaging Angles

A deep learning (DL)-based, deformable registration-driven liver tumor localization technique was developed for onboard deformable motion tracking. The technique (Surf-X360-Bio) uses in-room optical surface imaging and an x-ray projection at an arbitrary scan angle to solve volumetric liver and liver tumor motion in real-time. Surf-X360-Bio solves the volumetric motion of the liver and localizes the liver tumor, through deforming liver and liver tumor meshes segmented on prior 4D-CTs/CBCTs. It uses real-time onboard information from an optical surface image and a simultaneously-acquired x-ray projection (from an arbitrary scan angle). Surf-X360-Bio localizes tumors via two steps: liver boundary motion estimation and intra-liver motion derivation. Surf-X360-Bio first estimates liver boundary motion by a patient-specific surface imaging model (Surf), utilizing the correlation between the external body surface motion and the internal liver boundary motion. As the correlation can be imperfect, the residual motion estimation errors were corrected by a patient-specific, angle-agnostic x-ray imaging model (X360). X360 deformed the liver boundary to match motion-related imaging features encoded in an arbitrarily-angled x-ray projection, using the Surf output for initialization. X360 adopted a geometry-aware learning module to extract and adapt to angle-varying features of arbitrarily-angled x-ray images, by calculating the projection system matrix of each x-ray image on-the-fly during model training and inference. After the liver boundary motion estimation by Surf and X360, intra-liver deformation was solved by a biomechanical model (Bio) to propagate the liver boundary motion inside to localize the tumors. The DL-based Bio model used domain knowledge of tissue biomechanics and finite element analysis (FEA) to solve intra-liver motion, with a much faster speed than conventional FEA methods to meet the real-time constraint. Surf-X360-Bio was evaluated using a dataset of 34 liver patients. Liver tumor localization accuracy was evaluated by the Dice similarity coefficient (DSC), the 95-percentile Hausdorff distance (HD95), and the center-of-mass error (COME). Using 3,306 motion scenarios spanning the 360 degree x-ray scan angles for each testing patient, Surf-X360-Bio localized the liver tumors to 0.81 (mean) ± 0.16 (s.d.) in DSC, 2.5 ± 1.7 mm in HD95, and 2.1 ± 1.8 mm in COME. In comparison, the prior reference image without deformable registration-driven localization yielded 0.42 ± 0.29 in DSC, 8.1 ± 5.2 mm in HD95, and 8.5 ± 5.2 mm in COME. Via Surf-X360-Bio, the overall inference time was below 230 ms for each case. Combining optical surface imaging and x-ray imaging, Surf-X360-Bio achieved fast (<250 ms inference time), accurate (mean error < 2.1 mm), and robust liver tumor localizations at arbitrary x-ray scan angles for real-time, marker-less 3D deformable motion tracking.

Endoscope 3D reconstruction based on ORB-SLAM

Minimal invasive surgery (MIS) is the mainstream trend in developing surgical technology. As the endoscope is a significant tool in the surgical process, whether it can track the inner cavity and realize accurate real-time 3D reconstruction has a vital impact on the smooth progress of MIS. However, there are still problems in the endoscopic environment, such as severe image distortion, the effect of lighting conditions, and the inability to extract lumen textures. Orinted fast and rotated brief simultaneous localization and mapping (ORB-SLAM) is currently a relatively advanced simultaneous localization and mapping (SLAM) method with better performance. The ORB-SLAM-based endoscope 3D reconstruction method can improve performance and overcome the challenge of endoscope 3D reconstruction. This paper will first introduce several existing endoscope 3D reconstruction methods based on ORB-SLAM and analyze the limitations and issues of these methods through their experimental results. Then the paper will explore the solutions to the defects in this method from other methods and compare the characteristics and the result of experiments. Secondly, through the summary of the above methods and the introduction of the integration of the ORB-SLAM-based methods and other current advanced technologies, the future development trend and huge development potential of ORB-SLAM-based endoscopic 3D reconstruction are introduced. This paper will be of profound affection to further improve the optimization and application of the ORB-SLAM-based endoscope 3D reconstruction method.

Integration of Seismic Poisson Impedance Data with Real-Time Geomechanics and Real-Time 3D Ultradeep Resistivity Inversion Enabled New Opportunities in Developed Field: Case Study from Kuwait

Summary The Wara sandstone reservoir in the Minagish field of Kuwait Oil Company is a complex deposition of a typical pro-deltaic environment consisting of shaly-silty sandstone sequences, referenced as W7–W1. Three of these sequences (W6, W5, and W3) were expected in the case study well. The objective was to set a 9⅝-in. casing at the top of W6 and then drill through the Wara sequences to connect all of them and land and drill the lateral section within W3. The W6 sequence is typically the primary target in the Wara formation, being thick and consistent throughout the field. The next logical step in developing the Wara reservoir was to study and investigate the minor W5 and W3 members. Due to poor correlation of W5 and W3 channels in offset wells, the geological target was selected based on seismic Poisson impedance (PI). Historically, targeting the Wara formation occasionally resulted in multiple sidetracks due to drilling challenges. A real-time geomechanics service was used to overcome drilling challenges, and real-time 3D ultradeep resistivity inversion was implemented to optimize the well placement. An extensive predrilling study for geomechanical and ultradeep resistivity inversion modeling helped to develop a road map for an optimal and safe well-construction process. The study showed that use of real-time 3D ultradeep azimuthal resistivity (UDAR) inversion would help to optimize the well placement and maximize the sweet-zone exposure. The original well design, mud properties, and drilling parameters were modified based on the geomechanical study. Additionally, real-time geomechanics services were used to monitor and control the drilling process to follow the road map, which helped to avoid drilling issues, geostop at the W6 channel, and finally to run the casing smoothly. Real-time 3D ultradeep resistivity mapping in the lateral section helped the operator to drill through W6 and W5, land precisely, and drill the lateral in the W3 channel, which was well developed, as expected from seismic PI analysis. Formation evaluation of the lateral section showed an average porosity of 24 p.u., water saturation of 11%, and up to 3 darcies/cp mobility. The application of real-time 3D ultradeep resistivity inversion helped to triple the planned formation exposure and discover a geometric extension of the above deposited channels (W6 and W5), which will help for future field development. The flow test showed the highest production rates from W3 in the field. The integrated approach described above was recommended to be utilized for all future Wara wells.

Real-time localization and 3D semantic map reconstruction for unstructured citrus orchards

Semantic maps play a crucial role in smart agriculture, providing practical three-dimensional fruit tree data for orchard management and aiding the optimization of management strategies and improvement of economic benefits. However, previous map studies have mainly focused on geometric features and have lacked semantic information, limiting robots' ability to reason about useful information in complex tasks and achieve human–machine interaction. Furthermore, most existing map reconstructions have been offline or nonreal-time, making it difficult to satisfy the needs of real-time decision-making and planning in agricultural scenarios. Therefore, this paper proposes a real-time localization and semantic map reconstruction method for unstructured citrus orchards, integrating the visual-inertial SLAM VINS-RGBD framework with the semantic segmentation algorithm BiSeNetV1. By conducting semantic segmentation of 2D RGB images and mapping them to point clouds, a 3D semantic point cloud map is reconstructed. The statistical outlier removal filter and OctoMap are introduced for postprocessing to remove outliers and estimate obstacles in 3D space, constructing a more accurate, efficient and flexible map. The experimental results show that the proposed method achieved a semantic segmentation accuracy mIoU of 79.31% on a self-built citrus dataset, a citrus recall relative error of 11.29% and a localization accuracy mean translational error of 1.917 m with the map constructed under an unstructured orchard scenario. Additionally, the average memory saving rate of the statistical outlier removal filter was 10.36%, and the average memory saving rate of OctoMap was 97.39%. The processing time for each frame of real-time front-end feature detection and tracking was 11.14 ms. Moreover, the deployed semantic segmentation network BiSeNetV1 achieved a processing time of 7.35 ms per frame. These results indicate that the proposed method can achieve both high accuracy and real-time performance in semantic map reconstruction. This exploratory work provides theoretical and technical references for future research on more precise localization and more complete semantic mapping and has extensive application potential, providing essential technical support for intelligent agriculture.

Real-time 4D MRI using MR signature matching (MRSIGMA) on a 1.5T MR-Linac system.

Objective. To develop real-time 4D MRI using MR signature matching (MRSIGMA) for volumetric motion imaging in patients with pancreatic cancer on a 1.5T MR-Linac system.Approach. Two consecutive MRI scans with 3D golden-angle radial stack-of-stars acquisitions were performed on ten patients with inoperable pancreatic cancer. The complete first scan (905 angles) was used to compute a 4D motion dictionary including ten pairs of 3D motion images and signatures. The second scan was used for real-time imaging, where each angle (275 ms) was processed separately to match it to one of the dictionary entries. The complete second scan was also used to compute a 4D reference to assess motion tracking performance.Dicecoefficients of the gross tumor volume (GTV) and two organs-at-risk (duodenum-stomach and small bowel) were calculated between signature matching and reference. In addition, volume changes, displacements, center of mass shifts, andDicescores over time were calculated to characterize motion.Main results. Total imaging latency of MRSIGMA (acquisition + matching) was less than 300 ms. TheDicecoefficients were 0.87 ± 0.06 (GTV), 0.86 ± 0.05 (duodenum-stomach), and 0.85 ± 0.05 (small bowel), which indicate high accuracy (high mean value) and low uncertainty (low standard deviation) of MRSIGMA for real-time motion tracking. The center of mass shift was 3.1 ± 2.0 mm (GTV), 5.3 ± 3.0 mm (duodenum-stomach), and 3.4 ± 1.5 mm (small bowel). TheDicescores over time (0.97 ± [0.01-0.03]) were similarly high for MRSIGMA and reference scans in all the three contours.Significance. This work demonstrates the feasibility of real-time 4D MRI using MRSIGMA for volumetric motion tracking on a 1.5T MR-Linac system. The high accuracy and low uncertainty of real-time MRSIGMA is an essential step towards continuous treatment adaptation of tumors affected by real-time respiratory motion and could ultimately improve treatment safety by optimizing ablative dose delivery near gastrointestinal organs.

Aqua3DNet: Real-time 3D pose estimation of livestock in aquaculture by monocular machine vision

We present a low-cost monocular 3D position estimation method for perception in aquaculture monitoring. Video surveillance of aquaculture has many advantages but given the size of farms and the complexity of their habitats, it is not feasible for farmers to continuously monitor fish health. We formulate a novel end-to-end deep visual learning pipeline called Aqua3DNet that estimates fish pose using a bottom-up approach to detect and assign key features in one pass. In addition, a depth estimation model using Saliency Object Detection (SOD) masks is implemented to track the 3D position of the fish over time, which is used in this paper to create 3D density heat maps of the fish. The evaluation of the algorithm's performance shows that the detection accuracy reaches 80.63%, the F1 score reaches 87.34%, and the frames per second (fps) reaches 5.12. Aqua3DNet achieves comparable performance to other aquaculture-based computer vision and depth estimation models, with minimal decrease in speed despite the synthesis of the two models.

Reproducibility of transthoracic 3D echocardiography in the assessment of mitral valve area in patients with rheumatic mitral stenosis: real time versus ECG-gated 3D echocardiography.

To assess reproducibility of Real time 3D echocardiography (RT3D) and ECG-gated 3D echocardiography (EG3D) when measuring the mitral valve area (MVA) in rheumatic mitral stenosis (MS). MVA was assessed by three operators in 68 MS patients using RT3D and EG3D. Reproducibility of each technique was determined by calculating the standard error of measurements (SEM). SEM was similar between RT3D and EG3D. MVA variability was of 0.4cm² or 30% of any RT3D or EG3D measured MVA. The minimal change in MVA above which two measurements should be considered to differ significantly for the same operator was of 0.4cm² for RT3D and 0.5cm² for EG3D. For two different operators making successive measurements, the minimum significant change was of 0.5cm² for RT3D and 0.6cm² for EG3D. The minimum significant difference when switching from RT3D to EG3D or vice versa is of 0.6cm². Low temporal resolution of 6Hz has the least variability when using RT3D (0.19cm² vs. 0.26cm², p = 0.009) but significantly underestimated MVA (1.3 ± 0.4cm² vs. 1.4 ± 0.4cm², p < 10- 3) when compared to EG3D. MVA variability was significantly higher in mild MS when compared to severe MS whether it is RT3D (0.23cm² vs. 0.18cm², p = 0.02) or EG3D (0.27cm² vs. 0.16cm², p < 0.001). RT3D and EG3D are equally reproducible in the assessment of MVA in patients with MS. Further measurements standardization is required to have a clinically acceptable estimations of the true 3D MVA and minimal detectable differences.

Highly accelerated free-breathing real-time 2D flow imaging using compressed sensing and shared velocity encoding.

2D real-time (RT) phase-contrast (PC) MRI is a promising alternative to conventional PC MRI, which overcomes problems due to irregular heartbeats or poor respiratory control. This study aims to evaluate a prototype compressed sensing (CS)-accelerated 2D RT-PC MRI technique with shared velocity encoding (SVE) for accurate beat-to-beat flow measurements. The CS RT-PC technique was implemented using a single-shot fast RF-spoiled gradient echo with SVE by symmetric velocity encoding, and acquired with a temporal resolution of 51-56.5ms in 1-5 heartbeats. Both aortic dissection phantom (n = 8) and volunteer (n = 7) studies were conducted using the prototype CS RT (CS, R = 8), the conventional (GRAPPA, R = 2), and the fully sampled PC sequences on a 3T clinical system. Flow parameters including peak velocity, peak flow rate, net flow rate, and maximum velocity were calculated to compare the performance between different methods using linear regression, intraclass correlation (ICC), and Bland-Altman analyses. Comparisons of the flow measurements at all locations in the phantoms demonstrated an excellent correlation (all R2 ≥ 0.93) and agreement (all ICC ≥ 0.97) with negligible means of differences. In healthy volunteers, a similarly good correlation (all R2 ≥ 0.80) and agreement (all ICC ≥ 0.90) were observed; however, CS RT slightly underestimated the maximum velocities and flow rates (~ 12%). The highly accelerated CS RT-PC technique is feasible for the evaluation of flow patterns without requiring breath-holding, and it allows for rapid flow assessment in patients with arrhythmia or poor breath-hold capacity. The free-breathing real-time flow MRI technique offers improved spatial and temporal resolutions, as well as the ability to image individual cardiac cycles, resulting in superior image quality compared to the conventional PC technique when imaging patients with arrhythmias, especially those with atrial fibrillation. • The highly accelerated prototype CS RT-PC MRI technique with improved temporal resolution by the concept of SVE is feasible for beat-to-beat flow evaluation without requiring breath-holding. • The results of the phantom and in vivo quantitative flow evaluation show the ability of the prototype CS RT-PC technique to obtain reliable flow measurements similarly to the conventional PC MRI. • With less than 12% underestimation, excellent agreements between the two techniques were shown for the measurements of peak velocities and flow rates.

Three-dimensional Echocardiography during Percutaneous Mitral Balloon Valvuloplasty

Percutaneous mitral balloon valvuloplasty (PMBV) has been frequently used as an alternative to open or closed surgical mitral commissurotomy for the treatment of selected patients with symptomatic rheumatic mitral stenosis. PMBV has been proven a safe and effective method, often performed under fluoroscopic guidance. With the advent of three-dimensional echocardiography (3DE), interventional cardiologists have been able to successfully perform complex procedures such as paravalvular leak closure, transcatheter edge-to-edge repair of the mitral valve (MV) using MitraClip, etc. With the expertise derived from these procedures, 3DE has received considerable interests in many other procedures, including PMBV. Pre-, intra-, and postprocedural assessments of the mitral valve (MV) area; commissural splitting; the presence, degree, and location of mitral regurgitation; and other procedure-related complications can be more accurately and readily made using 3DE. In addition, real-time 3D transesophageal echocardiography has been proven to be more accurate than two-dimensional (2D) techniques in identifying specific MV abnormalities associated with complex diseases. Moreover, real-time 3DE offers the advantage of shorter fluoroscopy and radiation times throughout PMBV. This article reviews the use of 3DE during PMBV in light of the most recent literature and based on expertise and discusses the pros and cons of 3DE, with emphasis on its superiority to conventional 2D echocardiography.

Registration between 2D and 3D Ultrasound Images to Track Liver Blood Vessel Movement.

For the accurate positioning of surgical tools, conventional intraoperative navigation systems have been developed to recognize the relationship between target positions and the tools. However, since an internal organ is deformed during the operation, registration between realtime two-dimensional (2D) ultrasound images and three-dimensional (3D) CT or MRI images is not always effective. Therefore, this study developed image registration between 2D and 3D ultrasound images considering deformation for tracking target vessel movement in the liver. 3D ultrasound image was obtained in advance with 3D coordinates, including the target vessel. Then real-time 2D images and ultrasound probe position were simultaneously acquired using a 3D position sensor. We applied multiple image resolution registration, where rapid and fine optimizations can be expected at higher and lower levels, respectively. Meanwhile, the gradient descent method was adopted for the optimization, which determines the relative arrangements to obtain maximum similarity between 2D and 3D images. We experimentally established resolution level parameters using a phantom before applying it to track liver blood vessel movements in a normal healthy subject. Comparing the 2D images and the registered images, although the approach has some limitations in tracking large displacement, we confirmed that the cross-section of the target blood vessel was clearly visualized. This method has the potential for an ultrasound therapy targeting blood vessels under natural respiration conditions.

Tracking the flight and landing behaviour of western flower thrips in response to single and two-colour cues

Real-time 3D tracking and high-speed videography was used to examine the behaviour of a worldwide greenhouse pest, the western flower thrips (WFT), in response to different colours in the context of improving trap design. Measurements were taken of the number of landings on, and flight activity near, a lamp containing two LEDs of either the same colour or a combination of two colours presented side by side. Main findings show that landing patterns of WFT are different between colours, with landings on UV(+ red) as highly attractive stimulus being mostly distributed at the bottom half of the lamp, while for yellow also as very attractive and green as a ‘neutral’ stimulus, landings were clearly on the upper rim of the lamp. Additionally, a positive interaction with the UV-A(+ red) and yellow combination elicited the highest number of landings and flight time in front of the LED lamp. Conversely, a negative interaction was observed with decreased landings and flight time found for yellow when blue was present as the adjacent colour. Overall, differences between treatments were less obvious for flight times compared to number of landings, with tracking data suggesting that WFT might use different colours to orientate at different distances as they approach a visual stimulus.

Role of Fetal Cardiac MRI in Assessment of Congenital Heart Diseases in Comparison to Fetal Echocardiography

Introduction •Abnormalities of the cardiovascular system are the most common congenital diseases in the fetus and the first cause of infant mortality (Johnson et al. 2014). •Without a doubt, Echocardiography is the method of choice to visualize the fetal cardiac cardiovascular abnormalities. Unlike fetal echocardiography imaging, cardiovascular magnetic resonance (CMR) is relatively unaected by maternal and fetal conditions such as maternal obesity, uterine myoma, twins, oligohydramnios, fetal position and rib calcification, which particularly impair sonographic visualization of the fetal heart (Donofrio et al. 2014; Wielandner et al. 2013). •Fetal cardiac MR imaging is a novel MRI technique which can provide valuable information that could add to the prenatal diagnosis and evaluation of cardiac and most of extra-cardiac anomalies. The evaluation of pre-natal congenital cardiac anomalies is important for postnatal management. Aim of the work To highlight the advantage of MRI over fetal echocardiography in assessment of fetal congenital cardiac anomalies. Methods and material Twenty-six feti with suspected or diagnosed congenital cardiac anomalies. All cases underwent detailed history taking, fetal echocardiography and fetal cardiac MR. Study is approved by the ethics review board of Ain Shams University Hospital. Fetal echocardiography was done according to standard protocol and performed by suitable curvilinear probe. Then MRI of the fetal heart was done with uterine SURVEY_BFFE (balanced fast field echo), Black blood single shot -BH, white blood Cine, sBFFE_RLT DYN 3D(real time 3Ddynamic). Images were acquired in the transverse, four-chamber, shortaxis, coronal and oblique sagittal views. Findings of fetal echocardiography were compared with that of cardiac MRI. Results •Regarding the diagnostic performance of prenatal Echo and fetal cardiac MRI showed significant moderate agreement between the two modalities in the detection of different congenital cardiac anomalies, Kappa test: 0.500; p-value 0.021*. •Qualitative analysis: All cardiac structures were delineable by echocardiography and cardiac MRI. There was full agreement(κ = 1) between fetal echocardiography and cardiac MRI in evaluating the cardiac situs(24normal,1 levocardia, 1 ectopia cordis), aorta (20normal, 3 hypoplastic aorta,1 aortic stenosis,1 rt. Sided aortic arch and 1 small sized aorta), transverse cardiac diameter (22 normal, 4 cardiomegaly), foramen ovale (25 normal, 1 large single atrium), Good agreement was found between echocardiography and cardiac MRI for detection of position of the cardiac apex (23 normal, 2 dextrocardia and 1 mesocardia) •Fetal MRI had a significant role in detection of additional extracardiac anomalies in 13 cases. Conclusion fetal cardiac MR imaging as an adjunct to fetal echocardiography may provide valuable information that could add to the prenatal diagnosis and evaluation of cardiac and most of extra cardiac anomalies.

Utility of a 2D kinematic HASTE sequence in magnetic resonance imaging assessment of adjacent segment degeneration following anterior cervical discectomy and fusion.

To evaluate a dynamic half-Fourier acquired single turbo spin echo (HASTE) sequence following anterior cervical discectomy and fusion (ACDF) at the junctional level for adjacent segment degeneration comparing dynamic listhesis to radiographs and assessing dynamic cord contact and deformity during flexion-extension METHODS: Patients with ACDF referred for cervical spine MRI underwent a kinematic flexion-extension sagittal 2D HASTE sequence in addition to routine sequences. Images were independently reviewed by three radiologists for static/dynamic listhesis, and compared to flexion-extension radiographs. Blinded assessment of the HASTE sequence was performed for cord contact/deformity between neutral, flexion, and extension, to evaluate concordance between readers and inter-modality agreement. Inter-reader agreement for dynamic listhesis and impingement grade and inter-modality agreement for dynamic listhesis on MRI and radiographs was assessed using the kappa coefficient and percentage concordance. A total of 28 patients, mean age 60.2 years, were included. Mean HASTE acquisition time was 42 s. 14.3% demonstrated high grade dynamic stenosis (> grade 4) at the adjacent segment. There was substantial agreement for dynamic cord impingement with 70.2% concordance (kappa = 0.62). Concordance across readers for dynamic listhesis using HASTE was 81.0% (68/84) (kappa = 0.16) compared with 71.4% (60/84) (kappa = 0.40) for radiographs. Inter-modality agreement between flexion-extension radiographs and MRI assessment for dynamic listhesis across the readers was moderate (kappa = 0.41; 95% confidence interval: 0.16 to 0.67). A sagittal flexion-extension HASTE cine sequence provides substantial agreement between readers for dynamic cord deformity and moderate agreement between radiographs and MRI for dynamic listhesis. Degeneration of the adjacent segment with instability and myelopathy is one of the most common causes of pain and neurological deterioration requiring re-operation following cervical fusion surgery. • A real-time kinematic 2D sagittal HASTE flexion-extension sequence can be used to assess for dynamic listhesis, cervical cord, contact and deformity. • The additional kinematic cine sequence was well tolerated and the mean acquisition time for the 2D HASTE sequence was 42 s (range 31-44 s). • A sagittal flexion-extension HASTE cine sequence provides substantial agreement between readers for dynamic cord deformity and moderate agreement between radiographs and MRI for dynamic listhesis.

EXPRESS 3D GEOMECHANICAL MODELING FOR INPUT TO HYDRAULIC FRACTURING ON CARBONATE OILFIELDS OF BASHKORTOSTAN REPUBLIC

The work describes a method of real-time 3D modeling of mechanical properties over the entire field area for the purposes of hydraulic fracturing. The uniqueness of this modeling approach is associated with the prompt acquisition of mechanical properties along the well paths of the field, which is especially important in conditions of insufficient geomechanics data.&#x0D; Traditional three-dimensional geomechanical modeling is a time-consuming and resource-intensive process. The data often describe the mechanical properties and stress-strain state of the rocks within the current drilling or development sector of the field and do not take into account the interwell space, especially in poorly drilled areas of the field. The method of real-time 3D geomechanical modeling makes it possible to quantify values of minimum horizontal stress used in hydraulic fracturing simulators, as well as static and dynamic Young's modulus and Poisson's ratio over the entire field area using the optimal amount of resources and in a shorter time.&#x0D; Based on the built three-dimensional cubes of elastic properties and stresses, correct data were obtained in the inter-well space, their vertical and lateral changes were analyzed, and frac design data were obtained at any well drilled through the simulated targets of the field.&#x0D; This paper covers the Arlanskoye, Znamenskoye and Yugomashevskoye fields (Bashneft PJSC). The target targets are carbonate deposits: Kashira and Podolsk strata of the Arlanskoye field, Tournaisian strata of the Znamensky field and Bashkir-Vereisky strata of the Yugomashevsky field.&#x0D; For the carbonate rocks of the target targets, laboratory core studies were carried out to determine elastic properties, and stable regional core-core relationships were obtained in the intervals of lower and middle Carboniferous carbonate deposits.&#x0D; 1D geomechanical models were built in RN-SIGMA software 84 wells at the intervals of the study targets. In current 3D geological models are introduced and distributed values of Young's modulus, Poisson's coefficient and minimum horizontal stress gradient is calculated. Analysis of changes in geomechanical properties by area, for each study object was carried out and data were obtained to create frac designs for 15,000 wells.

Dynamic 3D phase-shifting profilometry based ona corner optical flow algorithm.

Real-time 3D reconstruction has been applied in many fields, calling for many ongoing efforts to improve the speed and accuracy of the used algorithms. Phase shifting profilometry based on the Lucas-Kanade optical flow method is a fast and highly precise method to construct and display the three-dimensional shape of objects. However, in this method, a dense optical flow calculation is required for the modulation image corresponding to the acquired deformed fringe pattern, which consumes a lot of time and affects the real-time performance of 3D reconstruction and display. Therefore, this paper proposes a dynamic 3D phase shifting profilometry based on a corner optical flow algorithm to mitigate this issue. Therein, the Harris corner algorithm is utilized to locate the feature points of the measured object, so that the optical flow needs to calculate for only the feature points which, greatly reduces the amount of calculation time. Both our experiments and simulations show that our method improves the efficiency of pixel matching by four times and 3D reconstruction by two times.

GPU parallelization of particulate matter concentration modeling in indoor environment with cellular automata framework

Nowadays, the CFD approaches for modeling turbulent airflow and particulate matter (PM) concentration distribution are matured despite they all suffer from heavy computational demands. Particularly, in indoor PM concentration modeling, a novel cellular automata (CA) approach in “Modeling particulate matter concentration in indoor environment with cellular automata framework” is developed to achieve almost the same accuracy with improved efficiency as the Eulerian approach. To further enhance its efficiency, this study proposes two parallelization procedures. With the mechanism parallelization, the four PM transport mechanisms (flow advection, turbulent diffusion, gravitational settling, and boundary deposition) are simulated simultaneously instead of sequentially. Besides, using the GPU-based cell parallelization by adopting OpenCL 2.1 under Nvidia CUDA, the execution of all the PM transport mechanisms is performed parallelly on GPUs instead of sequentially on the CPU. Three parallelized CA scenarios, i.e., the parallelized CA approach with only the mechanism parallelization, with only the GPU-based cell parallelization, and with both the two parallelization procedures, are evaluated through two indoor PM concentration experiments. The three parallelized CA scenarios are found to maintain the accuracy but enhance the efficiency by 174%–210%, 1780%–5730%, and 2427%–7695%, respectively. Thus, the GPU-based cell parallelization obtains more efficiency enhancement than the mechanism parallelization. Furthermore, despite the simulations are performed on an i9 PC with an Intel UHD Graphics 630 graphic card, the parallelized CA approach with both the two parallelization procedures can enhance its efficiency up to 24–77 times, proving its considerable potentials as a useful tool for real-time 3D indoor PM distribution modeling.