123 AI vs Expert: A Comparison of Rapid Visual IVC Collapsibility Assessment between POCUS Experts and a Deep Learning Algorithm

Abstract

We sought to create a deep learning (DL) algorithm to determine the degree of inferior vena cava (IVC) collapsibility in critically ill patients to enable novice point of care ultrasound (POCUS) providers. We utilized publicly available long short term Memory (LSTM) DL basic architecture which can track temporal changes and relationships in real-time video, to create an algorithm for ultrasound video analysis. The algorithm was trained on public domain IVC ultrasound videos to improve its ability to recognize changes in varied ultrasound video. A total of 220 IVC videos were utilized; 10% of the data was randomly used for cross correlation during training. Data was augmented through video rotation and manipulation to multiply effective training data quantity. After training, the algorithm was tested on the 50 new IVC ultrasound video obtained from public domain sources and not part of the data set used in training or cross validation. Fleiss’ kappa was calculated to compare level of agreement between the three POCUS experts and DL algorithm and POCUS experts. There was very good agreement between the 3 POCUS experts with kappa = 0.87. Agreement between experts and algorithm was good with alpha = 0.73. Our algorithm showed good agreement with POCUS experts in visually estimating degree of IVC collapsibility which has been shown in previously published studies to differentiate fluid responsive from fluid unresponsive septic shock patients. Such an algorithm could be adopted to run in real time on any ultrasound machine with a video output, easing the burden on novice POCUS users by limiting their task to obtaining and maintaining a sagittal proximal IVC view and allowing the AI to make real time determinations.