Echo Data Sets Research Articles

A Deep Learning Model with Axial Attention for Radar Echo Extrapolation

ABSTRACT Radar echo extrapolation is an important approach in precipitation nowcasting which utilizes historical radar echo images to predict future echo images. In this paper, we introduce the self-attention mechanism into Trajectory Gated Recurrent Unit (TrajGRU) model. Under the sequence-to-sequence framework, we have developed a novel convolutional recurrent neural network called Self-attention Trajectory Gated Recurrent Unit (SA-TrajGRU), which incorporates the self-attention mechanism. The SA-TrajGRU model which combines spatiotemporal variant structure in TrajGRU and self-attention module is simple and effective. We evaluate our approach on the Moving MNIST-2 dataset and the CIKM AnalytiCup 2017 radar echo dataset. The experimental results show that the performance of the proposed SA-TrajGRU model is comparable to other convolutional recurrent neural network models. HSS and CSI scores of the SA-TrajGRU model are higher than scores of other models under the radar echo threshold of 25 dBZ, indicating that the SA-TrajGRU model has the most accurate prediction results under this threshold.

TISE-LSTM: A LSTM model for precipitation nowcasting with temporal interactions and spatial extract blocks

Precipitation nowcasting has a profound impact on humanity and society, especially in areas with heavy rainfall, playing a central role in alerting against rainstorm disasters. At present, numerous deep learning-based methods have been proposed and proven superior to traditional radar echo extrapolation techniques like the Recurrent Neural Networks (RNNs). Our study introduces a novel precipitation forecasting model named TISE-LSTM, which can use the real image of the past half hour to predict the radar echo image of the next hour. By integrating the TIB and SEB into TISL-LSTM, we can alleviate the inherent issue observed in existing models, which is the decrease in forecast accuracy for high radar echo regions with extended extrapolation time. On both the CIKM23017 and AHEM real radar echo datasets, the performance of TISL-LSTM (including POD, FAR, CSI and HSS) is improved compared to the second-ranked model by 7.53%, 2.32%, 8.93%, 2.6%, and 14.84%, 6.18%, 8.92%, 18.12%, respectively, when the precipitation threshold is set to 40. Moreover, our model obtained an optimal MAE, MSE, SSIM score. Predicted images and the graphs of each metric over extrapolation time both demonstrate that our model accurately forecasts regions with high radar echoes even with a one-hour extrapolation.

MSLKNet: A Multi-Scale Large Kernel Convolutional Network for Radar Extrapolation

Radar echo extrapolation provides important information for precipitation nowcasting. Existing mainstream radar echo extrapolation methods are based on the Single-Input-Single-Output (SISO) architecture. These approaches of recursively predicting the predictive echo image with the current echo image as input often results in error accumulation, leading to severe performance degradation. In addition, the echo motion variations are extremely complex. Different regions of strong or weak echoes should receive different degrees of attention. Previous methods have not been specifically designed for this aspect. This paper proposes a new radar echo extrapolation network based entirely on a convolutional neural network (CNN). The network uses a Multi-Input-Multi-Output (MIMO) architecture to mitigate cumulative errors. It incorporates a multi-scale, large kernel convolutional attention module that enhances the extraction of both local and global information. This design results in improved performance while significantly reducing training costs. Experiments on dual-polarization radar echo datasets from Shijiazhuang and Nanjing show that the proposed fully CNN-based model can achieve better performance while reducing computational cost.

FastNet: A feature aggregation spatiotemporal network for predictive learning

Spatiotemporal prediction is a challenging topic because of uncertainty. Existing works attempt to design complicated systems to learn short-term and long-term dynamics. However, these models suffer from heavy computational burden on spatiotemporal data, especially for high-resolution frames. To reduce resource dependence, we propose FastNet, a novel and light encoder–decoder for predictive learning. We stack four ConvLSTM (Convolutional Long Short-Term Memory) based layers to construct a hierarchical framework. Based on this architecture, the feature aggregation module first aligns the temporal context, then decouples different frequency information, next gathers multi-level features and last synthesizes new feature maps alternately. We aggregate various and hierarchical features into predictions bringing two benefits: rich multi-level features and low resource usage. As for the unit blocks, depth-wise separable convolution is used to improve model efficiency and compress model size. Besides, we adopt perceptual loss as the cost function between ground truths and predictions, which helps our model to get higher similarity with true frames. In experiments, we evaluate the performances of FastNet on the MovingMNIST (Mixed National Institute of Standards and Technology) and Radar Echo datasets to verify its effectiveness. The quantitative metrics on the Radar Echo dataset show that FastNet reaches a slight increase in accuracy but up to 84% decline in computation compared with PredRNN-V2. Therefore, our FastNet achieves competitive results with lower resource usage and fewer parameters than the state-of-the-art model.

A Generative Adversarial and Spatiotemporal Differential Fusion Method in Radar Echo Extrapolation

As an important part of remote sensing data, weather radar plays an important role in convective weather forecasts to reduce extreme precipitation disasters. The existing radar echo extrapolation methods do not utilize the local natural characteristics of the radar echo effectively but only roughly extract the whole characteristics of the radar echo. To address these challenges, we design a spatiotemporal difference and generative adversarial fusion model (STDGAN). Specifically, a spatiotemporal difference module (STD) is designed to extract local weather patterns and model them in detail. In our model, spatiotemporal difference information and spatiotemporal features captured by the model itself are fused together. In addition, our model is trained in a generative adversarial network (GAN) framework; it helps to generate a clearer map of future radar echoes at the image level. The discriminator consists of multi-scale feature extractors, which can simulate weather models of various scales more completely. Finally, extrapolation experiments were conducted using actual radar echo data from Shijiazhuang and Nanjing. The experiments have shown that our model has a more accurate prediction performance for predicting local weather patterns and overall echo change trajectories compared with previous research models. Our model achieved MSE, PSNE, and SSIM values of 132.22, 37.87, and 0.796, respectively, on the Shijiazhuang radar echo dataset. In addition, our model also showed better performance results on the Nanjing radar echo dataset. The results show that the MSE was 49.570, the PSNR was 0.714, and the SSIM was 30.633. The CC value was 0.855.

Abstract 14699: Effects of Papillary Muscle Approximation and Annuloplasty Ring Sizing for Functional Mitral Regurgitation Repair in Enlarged Ventricles

Introduction: Our recent large animal study showed that papillary muscle approximation (PMA) repairs functional mitral regurgitation (FMR) and increases valve coaptation in enlarged left ventricles without negatively affecting ventricular function. However, since PMA does not reduce mitral annular dilation, addition of annuloplasty ring might be beneficial for such repair. In this study, we used computational modeling to investigate the benefit of the addition of annuloplasty to PMA and the effects of ring sizing on FMR repair in enlarged ventricles. Methods: 3D echo images were obtained in 3 pigs with heart failure, enlarged ventricles, and FMR. Each echo dataset was segmented, and computational model of the mitral valve was developed. To mimic PMA repair, the papillary muscles were drawn together in diastole. A true-sized annuloplasty ring of 40 mm and two downsized-rings of 38 mm and 36 mm were then added to each model. Leaflet closure was simulated and mitral valve geometry and biomechanics in peak systole were evaluated ( Fig A-B ). Results: On average, compared to pre-repair, PMA increased coaptation area by 30.6%, while the addition of annuloplasty increased this area by 85.1% with 40 mm, 138.1% with 38 mm, and 190.6% with 36 mm ring. Tenting area was reduced by 45.8% after PMA, 64.1% with the addition of 40 mm ring, 70.6% with 38 mm, and 76.5% with 36 mm ring. PMA reduced tension forces in chordae by 19.8%, while adding true-sized ring reduced forces by 16.4%. Annular downsizing allowed to reduce chordal forces by 31.2% with 38 mm, and 43.0% with 36 mm ring. Peak leaflet stress was observed on the P2 cusp, and PMA reduced it by 26.9%. True-sized ring reduced peak stress by only 10.7%, while 38 mm ring by 29.9%, and 36 mm by 43.7% ( Fig C ). Conclusions: Addition of true-sized ring to PMA further improved systolic valve geometry but not biomechanical parameters. Annular downsizing added to PMA was more beneficial for FMR repair, as it improved both mitral valve geometry and biomechanics.

GraphAT Net: A Deep Learning Approach Combining TrajGRU and Graph Attention for Accurate Cumulonimbus Distribution Prediction

In subtropical regions, heavy rains from cumulonimbus clouds can cause disasters such as flash floods and mudslides. The accurate prediction of cumulonimbus cloud distribution is crucial for mitigating such losses. Traditional machine learning approaches have been used on radar echo data generated by constant altitude plan position indicator (CAPPI) radar systems for predicting cumulonimbus cloud distribution. However, the results are often too foggy and fuzzy. This paper proposes a novel approach that integrates graph convolutional networks (GCN) and trajectory gated recurrent units (TrajGRU) with an attention mechanism to predict cumulonimbus cloud distribution from radar echo data. Experiments were conducted using the moving modified National Institute of Standards and Technology (moving MNIST) dataset and real-world radar echo data, and the proposed model showed a 59.12% improvement in mean square error (MSE) and a 16.26% improvement in structure similarity index measure (SSIM) on average in the moving MNIST dataset, a 65.40% improvement in MSE, and an 10.29% improvement in SSIM on average in the radar echo dataset. These results demonstrate the effectiveness of the proposed approach for improving the prediction accuracy of cumulonimbus cloud distribution.

CEMA-LSTM: Enhancing Contextual Feature Correlation for Radar Extrapolation Using Fine-Grained Echo Datasets

Accurate precipitation nowcasting can provide great convenience to the public so they can conduct corresponding arrangements in advance to deal with the possible impact of upcoming heavy rain. Recent relevant research activities have shown their concerns on various deep learning models for radar echo extrapolation, where radar echo maps were used to predict their consequent moment, so as to recognize potential severe convective weather events. However, these approaches suffer from an inaccurate prediction of echo dynamics and unreliable depiction of echo aggregation or dissipation, due to the size limitation of convolution filter, lack of global feature, and less attention to features from previous states. To address the problems, this paper proposes a CEMA-LSTM recurrent unit, which is embedded with a Contextual Feature Correlation Enhancement Block (CEB) and a Multi-Attention Mechanism Block (MAB). The CEB enhances contextual feature correlation and supports its model to memorize significant features for near-future prediction; the MAB uses a position and channel attention mechanism to capture global features of radar echoes. Two practical radar echo datasets were used involving the FREM and CIKM 2017 datasets. Both quantification and visualization of comparative experimental results have demonstrated outperformance of the proposed CEMA-LSTM over recent models, e.g., PhyDNet, MIM and PredRNN++, etc. In particular, compared with the second-ranked model, its average POD, FAR and CSI have been improved by 3.87%, 1.65% and 1.79%, respectively on the FREM, and by 1.42%, 5.60% and 3.16%, respectively on the CIKM 2017.

Radar Based Precipitation Nowcasting Prediction by Using Deep Learning Techniques

Nowcasting is an emerging area in meteorology that focuses on accurately anticipating the severity of short-term rainfall for a particular location. It is essential to many facets of society. Owing to its significance, researchers are experimenting to predict short term rainfall using neural network approaches. This study analyses proposes a novel method of merging Convolutional Neural Network and Long Short-Term Memory neural networks on a radar echo dataset. The model was tested against a synthetic moving mnist dataset before applying on actual radar image dataset. Given the previous radar images, the model could successfully find future image sequences and obtained an accuracy of more than 90%.

Velocity Estimation for Vehicle-Mounted SAR Based on Deep-Learning Framework

With the development of intelligent driving technology, vehicle-mounted synthetic aperture radar (SAR) imaging, which is regarded as one of the most fundamental assists, has become an important and focal topic. As for automotive SAR imaging, the primary task is to estimate the velocity of the moving platform. Due to the volatile and complex road conditions, it does expense tremendous efforts for accurate velocities acquired with additional motion sensors. In this article, based on exploiting the geometrical relations between the moving platform and near-ground echoes, a velocity estimation scheme is deduced with a single radar sensor only. Then, with the proposed geometric model, an accurate velocity can be obtained and quantitatively analyzed. In addition, to further improve the efficiency of velocity estimation, we establish a real-world SAR echo dataset and transform the conventional equipment-dependable problem into a data-driven method based on a deep neural network. The efficiency and effectiveness of the proposed methods are verified by SAR data collected from real-world scenes.

Abstract 10580: Machine Learning-Assisted Echocardiographic Identification of Children at High Risk for Treatment-Related Cardiomyopathy: A Proof-of-Concept Study

Introduction: Childhood cancer survivors require life-long surveillance for treatment-related cardiomyopathy/heart failure (CHF). Ultimately, we aim to use machine learning to aid in echocardiographic discrimination of survivors more likely to develop future CHF. For this feasibility pilot, we built two proof-of-concept deep convolutional neural networks (DCNNs) tasked with binary classification of present/future CHF versus no CHF with the aim of assessing optimal data input format and model architecture. Methods: From a robust multi-institutional surveillance echo dataset, we selected pilot data comprising 171 parasternal short axis echo clips, 52 from 5 CHF+ (present and future CHF) and 119 from 21 CHF- patients. We built two DCNN models differing in frame selection for input data (Fig. 1), both tasked with binary classification of CHF+ vs. CHF-. We used holdout subsets in a 10-fold cross-validation framework to test model performance and Student’s t test (paired) to compare model performance. Results: Classification performance was similar, with mean AUROC values of 0.59 ± 0.09 and 0.53 ± 0.11 (p=0.14) for the models trained on Type I and Type II montages, respectively (Fig. 2). Conclusions: The DCNN framework is feasible for a model tasked with classifying CHF status, and we are optimistic that a similar model can be trained with pre-CHF diagnosis (case) vs control patient images to facilitate machine learning-assisted identification of childhood cancer survivors more likely to develop CHF in the future.

Study on Small Samples Active Sonar Target Recognition Based on Deep Learning

Underwater target classification methods based on deep learning suffer from obvious model overfitting and low recognition accuracy in the case of small samples and complex underwater environments. This paper proposes a novel classification network (EfficientNet-S) based on EfficientNet-V2S. After optimization with model scaling, EfficientNet-S significantly improves the recognition accuracy of the test set. As deep learning models typically require very large datasets to train millions of model parameter, the number of underwater target echo samples is far more insufficient. We propose a deep convolutional generative adversarial network (SGAN) based on the idea of group padding and even-size convolution kernel for high-quality data augmentation. The results of anechoic pool experiments show that our algorithm effectively suppresses the overfitting phenomenon, achieves the best recognition accuracy of 92.5%, and accurately classifies underwater targets based on active echo datasets with small samples.

A self-attention integrated spatiotemporal LSTM approach to edge-radar echo extrapolation in the Internet of Radars

In recent years, the number of weather-related disasters significantly increases across the world. As a typical example, short-range extreme precipitation can cause severe flooding and other secondary disasters, which therefore requires accurate prediction of extent and intensity of precipitation in a relatively short period of time. Based on the echo extrapolation of networked weather radars (i.e., the Internet of Radars), different solutions have been presented ranging from traditional optical-flow methods to recent deep neural networks. However, these existing networks focus on local features of echo variations to model the dynamics of holistic radar echo motion, so it often suffers from inaccurate extrapolation of the radar echo motion trend, trajectory, and intensity. To address the problem, this paper introduces the self-attention mechanism and an extra memory that saves global spatiotemporal feature into the original Spatiotemporal LSTM (ST-LSTM) to form a self-attention Integrated ST-LSTM recurrent unit (SAST-LSTM), capturing both spatial and temporal global features of radar echo motion. And several these units are stacked to build the radar echo extrapolation network SAST-Net. Comparative experiments show that the proposed model has better performance on different real world radar echo datasets over other recent methods.

Accelerating Brain Imaging Using a Silent Spatial Encoding Axis.

PurposeTo characterize the acceleration capabilities of a silent head insert gradient axis that operates at the inaudible frequency of 20 kHz and a maximum gradient amplitude of 40 mT/m without inducing peripheral nerve stimulation.MethodsThe silent gradient axis' acquisitions feature an oscillating gradient in the phase‐encoding direction that is played out on top of a cartesian readout, similarly as done in Wave‐CAIPI. The additional spatial encoding fills k‐space in readout lanes allowing for the acquisition of fewer phase‐encoding steps without increasing aliasing artifacts. Fully sampled 2D gradient echo datasets were acquired both with and without the silent readout. All scans were retrospectively undersampled (acceleration factors R = 1 to 12) to compare conventional SENSE acceleration and acceleration using the silent gradient. The silent gradient amplitude and the readout bandwidth were varied to investigate the effect on artifacts and g‐factor.ResultsThe silent readout reduced the g‐factor for all acceleration factors when compared to SENSE acceleration. Increasing the silent gradient amplitude from 31.5 mT/m to 40 mT/m at an acceleration factor of 10 yielded a reduction in the average g‐factor (gavg) from 1.3 ± 0.14 (gmax = 1.9) to 1.1 ± 0.09 (gmax = 1.6). Furthermore, reducing the number of cycles increased the readout bandwidth and the g‐factor that reached gavg = 1.5 ± 0.16 for a readout bandwidth of 651 Hz/pixel and an acceleration factor of R = 8.ConclusionA silent gradient axis enables high acceleration factors up to R = 10 while maintaining a g‐factor close to unity (gavg = 1.1 and gmax = 1.6) and can be acquired with clinically relevant readout bandwidths.

An Advanced Scheme for Range Ambiguity Suppression of Spaceborne SAR Based on Blind Source Separation

Due to the minimum antenna area constrain of synthetic aperture radar (SAR), high-resolution and wide-swath (HRWS) imaging is difficult to be achieved using classical modes, such as scan, spotlight, and so on. Range ambiguity is one of the main technical challenges limiting HRWS imaging in spaceborne SAR. Insipred by the cocktail party effect, an advanced scheme based on blind source separation is put forward to suppress the range ambiguity of spaceborne SAR in this paper. In the scheme, multiple subantennas are used to collect multiple echo data sets, which are given different patterns. In this way, the echo signal of the desired region and that of the ambiguous region are weighted using different coefficients. During postprocessing, a detailed flow with the blind source separation algorithm is proposed to separate the desired echoes from those data sets. To verify the scheme, point target and scene simulations based on GF3 are carried out. The results indicate that the range ambiguity can be reduced by more than 15 dB using the proposed scheme, the processing flow is effective, and the scheme has a good performance without increasing the complexity of the system design under the current SAR system conditions. The scheme has the potential to be adopted in future spaceborne SAR missions.

Motion-Guided Global–Local Aggregation Transformer Network for Precipitation Nowcasting

Nowadays deep learning based weather radar echo extrapolation methods have competently improved nowcasting quality. Current pure convolutional or convolutional recurrent neural network based extrapolation pipelines inherently struggle in capturing both global and local spatiotemporal interactions simultaneously, thereby limiting nowcasting performances, e.g., they not only tend to underestimate heavy rainfalls’ spatial coverage and intensity but also fail to precisely predict non-linear motion patterns. Furthermore, the usually adopted pixel-wise objective functions lead to blurry predictions. To this end, we propose a novel motion-guided global-local aggregation Transformer network for effectively combining spatiotemporal cues at different time scales, thereby strengthening global-local spatiotemporal aggregation urgently required by the extrapolation task. First, we divide existing observations into both short and long term sequences to represent echo dynamics at different time scales. Then, to introduce reasonable motion guidance to Transformer, we customize an end-to-end module for jointly extracting Motion Representation of Short and Long term echo sequences (MRS, MRL), while estimating optical flow. Subsequently, based on Transformer architecture, MRS is used as queries to retrospect the most useful information from MRL for an effective aggregation of global long-term and local short-term cues. Finally, the fused feature is employed for future echo prediction. Additionally, for the blurry prediction problem, predictions from our model trained with an adversarial regularization achieve superior performances not only in nowcasting skill scores but also in precipitation details and image clarity over existing methods. Extensive experiments on two challenging radar echo datasets demonstrate the effectiveness of our proposed method.

A role for magnetic susceptibility in synthetic computed tomography.

Radiotherapy treatment planning based on magnetic resonance imaging (MRI) benefits from increased soft-tissue contrast and functional imaging. MRI-only planning is attractive but limited by the lack of electron density information required for dose calculation, and the difficulty to differentiate air and bone. MRI can map magnetic susceptibility to separate bone from air. A method is introduced to produce synthetic CT (sCT) through automatic voxel-wise assignment of CT numbers from an MRI dataset processed that includes magnetic susceptibility mapping. Volumetric multi-echo gradient echo datasets were acquired in the heads of five healthy volunteers and fourteen patients with cancer using a 3T MRI system. An algorithm for CT synthesis was designed using the volunteer data, based on fuzzy c-means clustering and adaptive thresholding of the MR data (magnitude, fat, water, and magnetic susceptibility). Susceptibility mapping was performed using a modified version of the iterative phase replacement algorithm. On patient data, the algorithm was assessed by direct comparison to X-ray computed tomography (CT) scans. The skull, spine, teeth, and major sinuses were clearly distinguished in all sCT, from healthy volunteers and patients. The mean absolute CT number error between X-ray CT and sCT in patients ranged from 78 and 134 HU. Susceptibility mapping using MRI can differentiate air and bone for CT synthesis. The proposed method is automated, fast, and based on a commercially available MRI pulse sequence. The method avoids registration errors and does not rely on a priori information, making it suitable for nonstandard anatomy.

Echo Time Dependency of Local Activity Metrics of Resting-State Functional MRI.

Local activity metrics of resting-state functional MRI (RS-fMRI), such as the amplitude of low-frequency fluctuation (ALFF), fractional ALFF (fALFF), regional homogeneity (ReHo), and degree centrality (DC), are widely used to detect brain abnormalities based on signal fluctuations. Although signal changes with echo time (TE) have been widely studied, the effect of TE on local activity metrics has not been investigated. RS-fMRI datasets from 12 healthy subjects with eyes open (EO) and eyes closed (EC) were obtained with a four-echo gradient-echo-planar imaging pulse sequence with the following parameters: repetition time/TE1/TE2/TE3/TE4 = 2,000/13/30.93/48.86/66.79 ms. Six representative regions were selected for simulating the spatial feature of TE dependency of local activity metrics. Moreover, whole-brain local activity metrics were calculated from each echo dataset and compared between EO and EC conditions. Dice overlap coefficient (DOC) was then employed to calculate the overlap between the T maps. We found that all the local activity metrics displayed different TE dependency characteristics, while their overall change patterns were similar: an initial large change followed by a slow variation. The T maps for local activity metrics also varied greatly with TE. For ALFF, fALFF, ReHo, and DC, the DOCs for voxels in four TE datasets were 6.87, 0.73, 5.08, and 0.93%, respectively. Collectively, these findings demonstrate that local metrics are greatly dependent on TE. Therefore, TE should be carefully considered for the optimization of data acquisition and multi-center data analysis in RS-fMRI.

A method of radar target detection based on convolutional neural network

Radar target detection (RTD) is one of the most significant techniques in radar systems, which has been widely used in the field of military and civilian. Although radar signal processing has been revolutionized since the introduction of deep learning, applying deep learning in RTD is considered as a novel concept. In this paper, we propose a model for multitask target detection based on convolutional neural network (CNN), which works directly with radar echo data and eliminates the need for time-consuming radar signal processing. The proposed detection method exploits time and frequency information simultaneously; therefore, the target can be detected and located in multi-dimensional space of range, velocity, azimuth and elevation. Due to the lack of labeled radar complex data, we construct a radar echo dataset with multiple signal-to-noise ratio (SNR) for RTD. Then, the CNN-based model is evaluated on the dataset. The experimental results demonstrated that the CNN-based detector has better detection performance and measuring accuracy in range, velocity, azimuth and elevation and more robust to noise in comparison with traditional radar signal processing approaches and other state-of-the-art methods.

Automatic orientation cues for intuitive immersive interrogation of 3D echocardiographic images in virtual reality using deep learning

Abstract Funding Acknowledgements Type of funding sources: Other. Main funding source(s): NIHR i4i funded 3D Heart Project Wellcome / EPSRC Centre for Medical Engineering (WT 203148/Z/16/Z) onbehalf 3D Heart Project Background/Introduction: In echocardiography (echo), image orientation is determined by the position and direction of the transducer during examination, unlike cardiovascular imaging modalities such as CT or MRI. As a result, when echo images are first shown their display orientation has no external anatomical landmarks, thus the user has to identify anatomical landmarks in the regions of interest to understand the orientation. Purpose To display an anatomical model of a standard heart, automatically aligned to an acquired patient’s 3D echo image - assisting image interpretation by quickly orienting the viewer. Methods 47 echo datasets from 13 pediatric patients with hypoplastic left heart syndrome (HLHS) were annotated by manually indicating the cardiac axes in both ES and ED volumes. We chose a view akin to the standard four chamber view in healthy hearts as the reference view, showing the AV valves, the right atrium, the left atrium and the hypoplastic ventricle. We then trained a deep convolutional neural network (CNN) to predict the rotation required for re-orientation to the reference view. Three data strategies were explored: 1) using 3D images to estimate orientation, 2) using three orthogonal slices only (2.5D approach) and 3) using the central slice only (2D approach). Three different algorithms were investigated: 1) an orientation classifier, 2) an orientation regressor with mean absolute angle error, and 3) an orientation regressor with geodesic loss. The data was split into training, validation and test sets with a 8:1:1 ratio. The training data was augmented by applying random rotations in the range [−10◦, +10◦] and updating labels accordingly. The model with smallest validation error was applied in tandem with the VR visualisation of the echo volumes. Results Experimental results suggest that a 2.5D CNN classifying discrete integer angles performs best in re-orienting volumetric images to the reference view, with a mean absolute angle error on the test set of 9.0 deg (test set error ranges from 10.8 to 25.9 deg. for other algorithms). An HLHS volumetric data (left) is automatically aligned with the cardiac model (right) using our trained network when loaded in VR as shown in Figure 1. The volume and the model are both cropped at the referencing plane. Conclusion A deep learning network to align 3D echo images to a reference view was successfully trained and then integrated into VR to reorient echo volumes to match a standard anatomical view. This work demonstrates the potential of combining artificial intelligence and VR in medical imaging, although further user study is expected to evaluate its clinical impact. Caption for Abstract Picture The VR user interface informs the user of the 3D echo image orientation, automatically aligning it with an anatomical model, here showing the four chamber apical view. Abstract Figure. Deep learning model integrated into VR