Echocardiogram Videos Research Articles

SimLVSeg: Simplifying Left Ventricular Segmentation in 2-D+Time Echocardiograms With Self- and Weakly Supervised Learning

ObjectiveAchieving reliable automatic left ventricle (LV) segmentation from echocardiograms is challenging due to the inherent sparsity of annotations in the dataset, as clinicians typically only annotate two specific frames for diagnostic purposes. Here we aim to address this challenge by introducing simplified LV segmentation (SimLVSeg), a novel paradigm that enables video-based networks for consistent LV segmentation from sparsely annotated echocardiogram videos. MethodsSimLVSeg consists of two training stages: (i) self-supervised pre-training with temporal masking, which involves pre-training a video segmentation network by capturing the cyclic patterns of echocardiograms from largely unannotated echocardiogram frames, and (ii) weakly supervised learning tailored for LV segmentation from sparse annotations. ResultsWe extensively evaluated SimLVSeg using EchoNet-Dynamic, the largest echocardiography dataset. SimLVSeg outperformed state-of-the-art solutions by achieving a 93.32% (95% confidence interval: 93.21–93.43%) dice score while being more efficient. We further conducted an out-of-distribution test to showcase SimLVSeg's generalizability on distribution shifts (CAM US dataset). ConclusionOur findings show that SimLVSeg exhibits excellent performance on LV segmentation with a relatively cheaper computational cost. This suggests that adopting video-based networks for LV segmentation is a promising research direction to achieve reliable LV segmentation. Our code is publicly available at https://github.com/BioMedIA-MBZUAI/SimLVSeg.

EFNet: A multitask deep learning network for simultaneous quantification of left ventricle structure and function

Purpose:The purpose of this study is to develop an automated method using deep learning for the reliable and precise quantification of left ventricle structure and function from echocardiogram videos, eliminating the need to identify end-systolic and end-diastolic frames. This addresses the variability and potential inaccuracies associated with manual quantification, aiming to improve the diagnosis and management of cardiovascular conditions. Methods:A single, fully automated multitask network, the EchoFused Network (EFNet) is introduced that simultaneously addresses both left ventricle segmentation and ejection fraction estimation tasks through cross-module fusion. Our proposed approach utilizes semi-supervised learning to estimate the ejection fraction from the entire cardiac cycle, yielding more dependable estimations and obviating the need to identify specific frames. To facilitate joint optimization, the losses from task-specific modules are combined using a normalization technique, ensuring commensurability on a comparable scale. Results:The assessment of the proposed model on a publicly available dataset, EchoNet-Dynamic, shows significant performance improvement, achieving an MAE of 4.35% for ejection fraction estimation and DSC values of 0.9309 (end-diastolic) and 0.9135 (end-systolic) for left ventricle segmentation. Conclusions:The study demonstrates the efficacy of EFNet, a multitask deep learning network, in simultaneously quantifying left ventricle structure and function through cross-module fusion on echocardiogram data.

Efficient deep learning-based automated diagnosis from echocardiography with contrastive self-supervised learning

BackgroundAdvances in self-supervised learning (SSL) have enabled state-of-the-art automated medical image diagnosis from small, labeled datasets. This label efficiency is often desirable, given the difficulty of obtaining expert labels for medical image recognition tasks. However, most efforts toward SSL in medical imaging are not adapted to video-based modalities, such as echocardiography.MethodsWe developed a self-supervised contrastive learning approach, EchoCLR, for echocardiogram videos with the goal of learning strong representations for efficient fine-tuning on downstream cardiac disease diagnosis. EchoCLR pretraining involves (i) contrastive learning, where the model is trained to identify distinct videos of the same patient, and (ii) frame reordering, where the model is trained to predict the correct of video frames after being randomly shuffled.ResultsWhen fine-tuned on small portions of labeled data, EchoCLR pretraining significantly improves classification performance for left ventricular hypertrophy (LVH) and aortic stenosis (AS) over other transfer learning and SSL approaches across internal and external test sets. When fine-tuning on 10% of available training data (519 studies), an EchoCLR-pretrained model achieves 0.72 AUROC (95% CI: [0.69, 0.75]) on LVH classification, compared to 0.61 AUROC (95% CI: [0.57, 0.64]) with a standard transfer learning approach. Similarly, using 1% of available training data (53 studies), EchoCLR pretraining achieves 0.82 AUROC (95% CI: [0.79, 0.84]) on severe AS classification, compared to 0.61 AUROC (95% CI: [0.58, 0.65]) with transfer learning.ConclusionsEchoCLR is unique in its ability to learn representations of echocardiogram videos and demonstrates that SSL can enable label-efficient disease classification from small amounts of labeled data.

ViViEchoformer: Deep Video Regressor Predicting Ejection Fraction.

Heart disease is the leading cause of death worldwide, and cardiac function as measured by ejection fraction (EF) is an important determinant of outcomes, making accurate measurement a critical parameter in PT evaluation. Echocardiograms are commonly used for measuring EF, but human interpretation has limitations in terms of intra- and inter-observer (or reader) variance. Deep learning (DL) has driven a resurgence in machine learning, leading to advancements in medical applications. We introduce the ViViEchoformer DL approach, which uses a video vision transformer to directly regress the left ventricular function (LVEF) from echocardiogram videos. The study used a dataset of 10,030 apical-4-chamber echocardiography videos from patients at Stanford University Hospital. The model accurately captures spatial information and preserves inter-frame relationships by extracting spatiotemporal tokens from video input, allowing for accurate, fully automatic EF predictions that aid human assessment and analysis. The ViViEchoformer's prediction of ejection fraction has a mean absolute error of 6.14%, a root mean squared error of 8.4%, a mean squared log error of 0.04, and an R 2 of 0.55. ViViEchoformer predicted heart failure with reduced ejection fraction (HFrEF) with an area under the curve of 0.83 and a classification accuracy of 87 using a standard threshold of less than 50% ejection fraction. Our video-based method provides precise left ventricular function quantification, offering a reliable alternative to human evaluation and establishing a fundamental basis for echocardiogram interpretation.

Opportunistic Screening of Chronic Liver Disease with Deep Learning Enhanced Echocardiography.

Chronic liver disease affects more than 1.5 billion adults worldwide, however the majority of cases are asymptomatic and undiagnosed. Echocardiography is broadly performed and visualizes the liver; but this information is not leveraged. To develop and evaluate a deep learning algorithm on echocardiography videos to enable opportunistic screening for chronic liver disease. Retrospective observational cohorts. Two large urban academic medical centers. Adult patients who received echocardiography and abdominal imaging (either abdominal ultrasound or abdominal magnetic resonance imaging) with ≤30 days between tests, between July 4, 2012, to June 4, 2022. Deep learning model predictions from a deep-learning computer vision pipeline that identifies subcostal view echocardiogram videos and detects the presence of cirrhosis or steatotic liver disease (SLD). Clinical diagnosis by paired abdominal ultrasound or magnetic resonance imaging (MRI). A total of 1,596,640 echocardiogram videos (66,922 studies from 24,276 patients) from Cedars-Sinai Medical Center (CSMC) were used to develop EchoNet-Liver, an automated pipeline that identifies high quality subcostal images from echocardiogram studies and detects the presence of cirrhosis or SLD. In the held-out CSMC test cohort, EchoNet-Liver was able to detect the presence of cirrhosis with an AUC of 0.837 (0.789 - 0.880) and SLD with an AUC of 0.799 (0.758 - 0.837). In a separate test cohort with paired abdominal MRIs, cirrhosis was detected with an AUC of 0.704 (0.689-0.718) and SLD was detected with an AUC of 0.726 (0.659-0.790). In an external test cohort of 106 patients (n = 5,280 videos), the model detected cirrhosis with an AUC of 0.830 (0.738 - 0.909) and SLD with an AUC of 0.768 (0.652 - 0.875). Deep learning assessment of clinical echocardiography enables opportunistic screening of SLD and cirrhosis. Application of this algorithm may identify patients who may benefit from further diagnostic testing and treatment for chronic liver disease.

Cross-modal interaction and multi-source visual fusion for video generation in fetal cardiac screening

To address the limitation of preserving data for dynamic visualization in fetal ultrasound screening, a novel framework is proposed to facilitate the generation of fetal four-chamber echocardiogram videos, incorporating multi-source visual fusion and understanding. The framework utilizes an effective spectrogram-ultrasound synchronizer to align the ultrasound images with time, ensuring the generated video matches the actual heartbeat rhythm. It further employs effective frame interpolation techniques to synthesize a video by incorporating a nonlinear bidirectional motion prediction. By integrating a Transformer model for the autoregressive generation of visual semantic sequence, the proposed framework demonstrates its capability to generate high-resolution frames. Experimental outcomes show the Clip-Similarity of 96.23% and DINOv2-Similarity of 99.77%. Furthermore, a multimodal dataset of fetal echocardiogram examinations has been constructed.

Deep learning in digital health with chatgpt: a study on efficient code generation

Abstract Background/Introduction Artificial Intelligence (AI) plays a significant role in the healthcare industry by providing innovative solutions to many health-related problems. Deep learning (DL) is a critical component in the development of AI-powered solutions; however, extensive knowledge is required to implement DL processes such as data preparation, model design, model training, and model evaluation. This has limited the widespread adoption of AI in the medical field, particularly in scenarios where non-technical professionals possess limited coding proficiency. Purpose This study evaluates the feasibility of using ChatGPT, a large language model developed by OpenAI, as an assistive tool for non-technical professionals to develop DL applications. The objective is to reduce the time and effort required in source code creation, while simplifying the overall process of application development. Methods ChatGPT was utilised to generate relevant source code for the creation of a DL application for echocardiogram view classification. Natural language input prompts were fed through the ChatGPT interface to produce modular scripts that fulfilled the purposes of video frame extraction, image pre-processing, dataset splitting, model training, and model evaluation. Input prompts were specified as a series of logical steps which explained the desired functionality, along with the necessary software libraries to be integrated. Results Experiments on code generation involved iteratively modifying input prompts through the rewording of sentences, reordering of steps, and usage of different libraries, to obtain working source code from ChatGPT responses. Results demonstrated that ChatGPT can generate fully usable programming scripts from natural language inputs, without the need for any modifications, to accomplish the various tasks involved in the DL process flow. Experiments were also conducted on locally sourced dobutamine stress echocardiogram videos, to verify script functionality and output reliability. The scripts successfully extracted, pre-processed, and split the video frames into training, validation, and testing sets. These datasets were then employed in the training and evaluation of a convolutional neural network which achieved an accuracy of above 80% when classifying seven different echocardiogram views. Conclusion ChatGPT has the potential to significantly reduce the time and effort required in developing DL applications. The successful implementation of the programming scripts suggest that ChatGPT can function as an assisting tool for non-technical professionals. Its ability to generate operational source code and streamline the development process could facilitate greater adoption of AI in healthcare and other industries. Future works shall explore the usage of ChatGPT in generating solutions to more heart-related use cases.Code generation and DL process flow

Abstract 16836: A View-Invariant Deep Learning Model for Estimating Ejection Fraction From Any Valid Echocardiogram Videoclip, Including Point-of-Care Ultrasound (POCUS)

Background: Recently, our team developed an end-to-end artificial intelligence (AI) framework that can successfully estimate left ventricular ejection fraction (LVEF) from echocardiogram videos. However, this framework requires specific views (A2C and PLAX), and cannot estimate LVEF for studies from which these views are missing. Goal: To develop a deep learning model to estimate LVEF from any type and number of echocardiogram video(s). Methods: We built a deep learning model for B-mode echocardiogram videos of the left ventricle (A2C, A3C, A4C, or PLAX views), using retrospective transthoracic echocardiography (TTE) data. The model was evaluated with two test datasets: one with random valid (i.e., showing the left ventricle) views, and one with preselected views (both having 433 patients). Later, the model was tested with a prospective patient cohort that had both TTE and point-of-care ultrasound (POCUS) data collected simultaneously (393 patients). Results: On the retrospective TTE test set, we observed excellent performance (five random valid views per study: RMSE of 6.87% and Pearson correlation (PC) of 0.82 on LVEF estimation, and an AUC of 0.96 (95% CI: of 0.94-0.98) on low LVEF classification (<=40%); preselected views: RMSE of 6.75%, PC of 0.83, and an AUC of 0.96 (95% CI: 0.94-0.98)). For both TTE and POCUS data from the prospective cohort, we also observed strong performance (TTE: RMSE of 5.62%, PC of 0.87, and an AUC of 0.97; POCUS: RMSE of 6.12%, PC of 0.84, and an AUC of 0.96). For the prospective test sets as well as for the retrospective TTE test set with preselected views, results reflect the estimated LVEF average of A2C, A3C, A4C, and PLAX videos from each patient. Conclusions: For both TTE and POCUS data, the current model showed strong performance which was comparable to that of our previously established view-specific model. Alongside this comparable performance, the current model surpasses our previous model in terms of flexibility, as it isn't dependent on view classification and allows a broader selection in terms of the number and types of videos per study or patient. This innovative approach could prove particularly beneficial in point-of-care situations in which it may not be feasible to obtain high-quality, view-specific videos.

Automated Echocardiographic Detection of Heart Failure With Preserved Ejection Fraction Using Artificial Intelligence

BackgroundDetection of heart failure with preserved ejection fraction (HFpEF) involves integration of multiple imaging and clinical features which are often discordant or indeterminate. ObjectivesThe authors applied artificial intelligence (AI) to analyze a single apical 4-chamber transthoracic echocardiogram video clip to detect HFpEF. MethodsA 3-dimensional convolutional neural network was developed and trained on apical 4-chamber video clips to classify patients with HFpEF (diagnosis of heart failure, ejection fraction ≥50%, and echocardiographic evidence of increased filling pressure; cases) vs without HFpEF (ejection fraction ≥50%, no diagnosis of heart failure, normal filling pressure; controls). Model outputs were classified as HFpEF, no HFpEF, or nondiagnostic (high uncertainty). Performance was assessed in an independent multisite data set and compared to previously validated clinical scores. ResultsTraining and validation included 2,971 cases and 3,785 controls (validation holdout, 16.8% patients), and demonstrated excellent discrimination (area under receiver-operating characteristic curve: 0.97 [95% CI: 0.96-0.97] and 0.95 [95% CI: 0.93-0.96] in training and validation, respectively). In independent testing (646 cases, 638 controls), 94 (7.3%) were nondiagnostic; sensitivity (87.8%; 95% CI: 84.5%-90.9%) and specificity (81.9%; 95% CI: 78.2%-85.6%) were maintained in clinically relevant subgroups, with high repeatability and reproducibility. Of 701 and 776 indeterminate outputs from the Heart Failure Association-Pretest Assessment, Echocardiographic and Natriuretic Peptide Score, Functional Testing (HFA-PEFF), and Final Etiology and Heavy, Hypertensive, Atrial Fibrillation, Pulmonary Hypertension, Elder, and Filling Pressure (H2FPEF) scores, the AI HFpEF model correctly reclassified 73.5% and 73.6%, respectively. During follow-up (median: 2.3 [IQR: 0.5-5.6] years), 444 (34.6%) patients died; mortality was higher in patients classified as HFpEF by AI (HR: 1.9 [95% CI: 1.5-2.4]). ConclusionsAn AI HFpEF model based on a single, routinely acquired echocardiographic video demonstrated excellent discrimination of patients with vs without HFpEF, more often than clinical scores, and identified patients with higher mortality.

Contour Tracking and detection of heart chambers in an echocardiogram video sequence using UNET

Contour Tracking and detection of heart chambers in an echocardiogram video sequence using UNET

Cyclical Self-Supervision for Semi-Supervised Ejection Fraction Prediction From Echocardiogram Videos.

Left-ventricular ejection fraction (LVEF) is an important indicator of heart failure. Existing methods for LVEF estimation from video require large amounts of annotated data to achieve high performance, e.g. using 10,030 labeled echocardiogram videos to achieve mean absolute error (MAE) of 4.10. Labeling these videos is time-consuming however and limits potential downstream applications to other heart diseases. This paper presents the first semi-supervised approach for LVEF prediction. Unlike general video prediction tasks, LVEF prediction is specifically related to changes in the left ventricle (LV) in echocardiogram videos. By incorporating knowledge learned from predicting LV segmentations into LVEF regression, we can provide additional context to the model for better predictions. To this end, we propose a novel Cyclical Self-Supervision (CSS) method for learning video-based LV segmentation, which is motivated by the observation that the heartbeat is a cyclical process with temporal repetition. Prediction masks from our segmentation model can then be used as additional input for LVEF regression to provide spatial context for the LV region. We also introduce teacher-student distillation to distill the information from LV segmentation masks into an end-to-end LVEF regression model that only requires video inputs. Results show our method outperforms alternative semi-supervised methods and can achieve MAE of 4.17, which is competitive with state-of-the-art supervised performance, using half the number of labels. Validation on an external dataset also shows improved generalization ability from using our method.

Bayesian Inference and Dynamic Neural Feedback Promote the Clinical Application of Intelligent Congenital Heart Disease Diagnosis

Congenital heart disease (CHD) is the leading cause of infant death. An artificial intelligence (AI)-based CHD diagnosis network (CHDNet) is an echocardiogram video-based binary classification model that judges whether echocardiogram videos contain heart defects. Existing CHDNets have shown performances comparable to or even better than medical experts, but their unreliability on cases outside of the training set has become the main bottleneck for their deployment. This is a common problem for most AI-based diagnostic approaches. Here, to overcome this challenge, we present two essential mechanisms—Bayesian inference and dynamic neural feedback—to respectively measure and improve the diagnostic reliability of AI. The former easily makes the neural network output its reliability instead of a single prediction result, while the latter is a computational neural feedback cell that allows the neural network to feed knowledge from the output layer back to the shallow layers and enables the neural network to selectively activate relevant neurons. To evaluate the effectiveness of these two mechanisms, we trained CHDNets on 4151 echocardiogram videos containing three common CHD defects and tested them on an internal test set of 1037 echocardiogram videos and an external set of 692 videos that were newly collected from other cardiovascular imaging devices. Each echocardiogram video corresponds to a unique patient and a unique visit. We demonstrate on various neural network architectures how the reliability obtained by Bayesian inference interprets and quantifies the significant performance difference between internal and external test sets of neural networks, and how the devised feedback cell helps the neural networks to maintain high accuracy and reliability, despite the input being corrupted by noise or when using an external test set.

AI-ENABLED ASSESSMENT OF CARDIAC FUNCTION AND VIDEO QUALITY IN EMERGENCY DEPARTMENT POINT-OF-CARE ECHOCARDIOGRAMS

BackgroundThe adoption of point-of-care ultrasound (POCUS) has greatly improved the ability to rapidly evaluate unstable emergency department (ED) patients at the bedside. One major use of POCUS is to obtain echocardiograms to assess cardiac function. ObjectivesWe developed EchoNet-POCUS, a novel deep learning system, to aid emergency physicians (EPs) in interpreting POCUS echocardiograms and to reduce operator-to-operator variability. MethodsWe collected a new dataset of POCUS echocardiogram videos obtained in the ED by EPs and annotated the cardiac function and quality of each video. Using this dataset, we train EchoNet-POCUS to evaluate both cardiac function and video quality in POCUS echocardiograms. ResultsEchoNet-POCUS achieves an area under the receiver operating characteristic curve (AUROC) of 0.92 (0.89–0.94) for predicting whether cardiac function is abnormal and an AUROC of 0.81 (0.78–0.85) for predicting video quality. ConclusionsEchoNet-POCUS can be applied to bedside echocardiogram videos in real time using commodity hardware, as we demonstrate in a prospective pilot study.

Automatic Diagnostic Tool for Detection of Regional Wall Motion Abnormality from Echocardiogram

The echocardiogram is an ultrasound imaging modality, employed to assess cardiac abnormalities. The Regional Wall Motion Abnormality (RWMA) is the occurrence of abnormal or absent contractility of a region of the heart muscle. Conventional assessment of RWMA is based on visual interpretation of endocardial excursion and myocardial thickening from the echocardiogram videos. Wall motion assessment accuracy depends on the experience of the sonographer. Current automated methods highly depend on the preprocessing steps such as segmentation of ventricle part or manually finding systole and diastole frames from an echocardiogram. Additionally, state-of-the-art methods majorly make use of images rather than videos, which specifically lack the usage of temporal information associated with an echocardiogram. The deep learning models used, employ highly complex networks with billions of trainable parameters. Further, the existing models used on video data add to the computational intensity because of the high frame rates of echocardiogram videos. We developed a novel deep learning architecture EC3D-Net (Echo-Cardio 3D Net), which captures the temporal information for identifying regional wall motion abnormality from echocardiogram. We demonstrate that EC3D-Net can extract temporal information from even raw echocardiogram videos, at low frame rates, employing minimal training parameter-based deep architecture. EC3D-Net achieves both an overall F1-Score and an Area Under Curve (AUC) score of 0.82. Further, we were able to reduce time for training and trainable parameters by 50% through minimizing frames per second. We also show the EC3D-Net is an interpretable model, thereby helping physicians understand our model prediction. RWMA detection from echocardiogram videos is a challenging process and our results demonstrate that we could achieve the state-of-the-art results even while using minimal parameters and time by our EC3D-Net. The proposed network outperforms both complex deep networks as well as fusion methods generally used in video classification.

Myocardial strain analysis of echocardiography based on deep learning.

Strain analysis provides more thorough spatiotemporal signatures for myocardial contraction, which is helpful for early detection of cardiac insufficiency. The use of deep learning (DL) to automatically measure myocardial strain from echocardiogram videos has garnered recent attention. However, the development of key techniques including segmentation and motion estimation remains a challenge. In this work, we developed a novel DL-based framework for myocardial segmentation and motion estimation to generate strain measures from echocardiogram videos. Three-dimensional (3D) Convolutional Neural Network (CNN) was developed for myocardial segmentation and optical flow network for motion estimation. The segmentation network was used to define the region of interest (ROI), and the optical flow network was used to estimate the pixel motion in the ROI. We performed a model architecture search to identify the optimal base architecture for motion estimation. The final workflow design and associated hyperparameters are the result of a careful implementation. In addition, we compared the DL model with a traditional speck tracking algorithm on an independent, external clinical data. Each video was double-blind measured by an ultrasound expert and a DL expert using speck tracking echocardiography (STE) and DL method, respectively. The DL method successfully performed automatic segmentation, motion estimation, and global longitudinal strain (GLS) measurements in all examinations. The 3D segmentation has better spatio-temporal smoothness, average dice correlation reaches 0.82, and the effect of target frame is better than that of previous 2D networks. The best motion estimation network achieved an average end-point error of 0.05 ± 0.03 mm per frame, better than previously reported state-of-the-art. The DL method showed no significant difference relative to the traditional method in GLS measurement, Spearman correlation of 0.90 (p < 0.001) and mean bias -1.2 ± 1.5%. In conclusion, our method exhibits better segmentation and motion estimation performance and demonstrates the feasibility of DL method for automatic strain analysis. The DL approach helps reduce time consumption and human effort, which holds great promise for translational research and precision medicine efforts.

Deep Learning for Automated Echocardiogram Analysis

In the fight of heart disease (the #1 global killer), Left ventricular ejection fraction (EF) calculated via echocardiography plays crucial role for disease diagnosis and treatment because EF can distinguish heart failure from normal cardiac function. However, traditional EF calculation is a time-consuming manual process with high variability ‒ a single routine echocardiogram generates about 10–50 videos (~3,000 images), coupled with the limited capacity to analyze such large dataset by human experts often results in misdiagnosis. This project aimed to develop transparent and interpretable deep learning pipeline for automated echocardiogram analysis to calculate EF for quick assessment of cardiac function. Using the EchoNet dataset, five PyTorch deep learning models were trained to automatically calculate EF achieving mean error rates that ranged from 14-16%, comparable to that of expert cardiac sonographers, and significantly outperformed qualitative analysis by physicians (~30% error rate). Among the five models, MobileNet was identified as the best deep learning model for web application and portable devices; therefore, it was deployed as a web-based app through AWS, standalone PC, and Raspberry Pi, enabling upload and analyze echocardiogram videos and obtain EF calculation results within seconds. Such automated echocardiogram analysis can dramatically speed up image analysis, reduce the burden on cardiologists, eliminate inter-observer variability, hence democratize echocardiography by enabling non-experts to quickly and accurately assess cardiac functions at point of care even in cardiology expertise limited rural areas and developing countries. Future work includes improving these models with additional data and adapting the app for handheld ultrasound devices.

Multimodal deep learning enhances diagnostic precision in left ventricular hypertrophy.

Determining the aetiology of left ventricular hypertrophy (LVH) can be challenging due to the similarity in clinical presentation and cardiac morphological features of diverse causes of disease. In particular, distinguishing individuals with hypertrophic cardiomyopathy (HCM) from the much larger set of individuals with manifest or occult hypertension (HTN) is of major importance for family screening and the prevention of sudden death. We hypothesized that an artificial intelligence method based joint interpretation of 12-lead electrocardiograms and echocardiogram videos could augment physician interpretation. We chose not to train on proximate data labels such as physician over-reads of ECGs or echocardiograms but instead took advantage of electronic health record derived clinical blood pressure measurements and diagnostic consensus (often including molecular testing) among physicians in an HCM centre of excellence. Using more than 18 000 combined instances of electrocardiograms and echocardiograms from 2728 patients, we developed LVH-fusion. On held-out test data, LVH-fusion achieved an F1-score of 0.71 in predicting HCM, and 0.96 in predicting HTN. In head-to-head comparison with human readers LVH-fusion had higher sensitivity and specificity rates than its human counterparts. Finally, we use explainability techniques to investigate local and global features that positively and negatively impact LVH-fusion prediction estimates providing confirmation from unsupervised analysis the diagnostic power of lateral T-wave inversion on the ECG and proximal septal hypertrophy on the echocardiogram for HCM. These results show that deep learning can provide effective physician augmentation in the face of a common diagnostic dilemma with far reaching implications for the prevention of sudden cardiac death.

Adaptive Contrast for Image Regression in Computer-Aided Disease Assessment.

Image regression tasks for medical applications, such as bone mineral density (BMD) estimation and left-ventricular ejection fraction (LVEF) prediction, play an important role in computer-aided disease assessment. Most deep regression methods train the neural network with a single regression loss function like MSE or L1 loss. In this paper, we propose the first contrastive learning framework for deep image regression, namely AdaCon, which consists of a feature learning branch via a novel adaptive-margin contrastive loss and a regression prediction branch. Our method incorporates label distance relationships as part of the learned feature representations, which allows for better performance in downstream regression tasks. Moreover, it can be used as a plug-and-play module to improve performance of existing regression methods. We demonstrate the effectiveness of AdaCon on two medical image regression tasks, i.e., bone mineral density estimation from X-ray images and left-ventricular ejection fraction prediction from echocardiogram videos. AdaCon leads to relative improvements of 3.3% and 5.9% in MAE over state-of-the-art BMD estimation and LVEF prediction methods, respectively.

Assessment of Artificial Intelligence in Echocardiography Diagnostics in Differentiating Takotsubo Syndrome From Myocardial Infarction

Machine learning algorithms enable the automatic classification of cardiovascular diseases based on raw cardiac ultrasound imaging data. However, the utility of machine learning in distinguishing between takotsubo syndrome (TTS) and acute myocardial infarction (AMI) has not been studied. To assess the utility of machine learning systems for automatic discrimination of TTS and AMI. This cohort study included clinical data and transthoracic echocardiogram results of patients with AMI from the Zurich Acute Coronary Syndrome Registry and patients with TTS obtained from 7 cardiovascular centers in the International Takotsubo Registry. Data from the validation cohort were obtained from April 2011 to February 2017. Data from the training cohort were obtained from March 2017 to May 2019. Data were analyzed from September 2019 to June 2021. Transthoracic echocardiograms of 224 patients with TTS and 224 patients with AMI were analyzed. Area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity of the machine learning system evaluated on an independent data set and 4 practicing cardiologists for comparison. Echocardiography videos of 228 patients were used in the development and training of a deep learning model. The performance of the automated echocardiogram video analysis method was evaluated on an independent data set consisting of 220 patients. Data were matched according to age, sex, and ST-segment elevation/non-ST-segment elevation (1 patient with AMI for each patient with TTS). Predictions were compared with echocardiographic-based interpretations from 4 practicing cardiologists in terms of sensitivity, specificity, and AUC calculated from confidence scores concerning their binary diagnosis. In this cohort study, apical 2-chamber and 4-chamber echocardiographic views of 110 patients with TTS (mean [SD] age, 68.4 [12.1] years; 103 [90.4%] were female) and 110 patients with AMI (mean [SD] age, 69.1 [12.2] years; 103 [90.4%] were female) from an independent data set were evaluated. This approach achieved a mean (SD) AUC of 0.79 (0.01) with an overall accuracy of 74.8 (0.7%). In comparison, cardiologists achieved a mean (SD) AUC of 0.71 (0.03) and accuracy of 64.4 (3.5%) on the same data set. In a subanalysis based on 61 patients with apical TTS and 56 patients with AMI due to occlusion of the left anterior descending coronary artery, the model achieved a mean (SD) AUC score of 0.84 (0.01) and an accuracy of 78.6 (1.6%), outperforming the 4 practicing cardiologists (mean [SD] AUC, 0.72 [0.02]) and accuracy of 66.9 (2.8%). In this cohort study, a real-time system for fully automated interpretation of echocardiogram videos was established and trained to differentiate TTS from AMI. While this system was more accurate than cardiologists in echocardiography-based disease classification, further studies are warranted for clinical application.

High-Throughput Precision Phenotyping of Left Ventricular Hypertrophy With Cardiovascular Deep Learning

Early detection and characterization of increased left ventricular (LV) wall thickness can markedly impact patient care but is limited by under-recognition of hypertrophy, measurement error and variability, and difficulty differentiating causes of increased wall thickness, such as hypertrophy, cardiomyopathy, and cardiac amyloidosis. To assess the accuracy of a deep learning workflow in quantifying ventricular hypertrophy and predicting the cause of increased LV wall thickness. This cohort study included physician-curated cohorts from the Stanford Amyloid Center and Cedars-Sinai Medical Center (CSMC) Advanced Heart Disease Clinic for cardiac amyloidosis and the Stanford Center for Inherited Cardiovascular Disease and the CSMC Hypertrophic Cardiomyopathy Clinic for hypertrophic cardiomyopathy from January 1, 2008, to December 31, 2020. The deep learning algorithm was trained and tested on retrospectively obtained independent echocardiogram videos from Stanford Healthcare, CSMC, and the Unity Imaging Collaborative. The main outcome was the accuracy of the deep learning algorithm in measuring left ventricular dimensions and identifying patients with increased LV wall thickness diagnosed with hypertrophic cardiomyopathy and cardiac amyloidosis. The study included 23 745 patients: 12 001 from Stanford Health Care (6509 [54.2%] female; mean [SD] age, 61.6 [17.4] years) and 1309 from CSMC (808 [61.7%] female; mean [SD] age, 62.8 [17.2] years) with parasternal long-axis videos and 8084 from Stanford Health Care (4201 [54.0%] female; mean [SD] age, 69.1 [16.8] years) and 2351 from CSMS (6509 [54.2%] female; mean [SD] age, 69.6 [14.7] years) with apical 4-chamber videos. The deep learning algorithm accurately measured intraventricular wall thickness (mean absolute error [MAE], 1.2 mm; 95% CI, 1.1-1.3 mm), LV diameter (MAE, 2.4 mm; 95% CI, 2.2-2.6 mm), and posterior wall thickness (MAE, 1.4 mm; 95% CI, 1.2-1.5 mm) and classified cardiac amyloidosis (area under the curve [AUC], 0.83) and hypertrophic cardiomyopathy (AUC, 0.98) separately from other causes of LV hypertrophy. In external data sets from independent domestic and international health care systems, the deep learning algorithm accurately quantified ventricular parameters (domestic: R2, 0.96; international: R2, 0.90). For the domestic data set, the MAE was 1.7 mm (95% CI, 1.6-1.8 mm) for intraventricular septum thickness, 3.8 mm (95% CI, 3.5-4.0 mm) for LV internal dimension, and 1.8 mm (95% CI, 1.7-2.0 mm) for LV posterior wall thickness. For the international data set, the MAE was 1.7 mm (95% CI, 1.5-2.0 mm) for intraventricular septum thickness, 2.9 mm (95% CI, 2.4-3.3 mm) for LV internal dimension, and 2.3 mm (95% CI, 1.9-2.7 mm) for LV posterior wall thickness. The deep learning algorithm accurately detected cardiac amyloidosis (AUC, 0.79) and hypertrophic cardiomyopathy (AUC, 0.89) in the domestic external validation site. In this cohort study, the deep learning model accurately identified subtle changes in LV wall geometric measurements and the causes of hypertrophy. Unlike with human experts, the deep learning workflow is fully automated, allowing for reproducible, precise measurements, and may provide a foundation for precision diagnosis of cardiac hypertrophy.