Quantitative liver fat fraction measurement by multi-view sonography using deep learning and attention maps

Abstract

Qualitative sonography is used to assess nonalcoholic fatty liver disease (NAFLD), an important health issue worldwide. We used B-mode image deep-learning to objectively assess NAFLD in 4 views of the liver (hepatic veins at confluence with inferior vena cava, right portal vein, right posterior portal vein and liver/kidney) in 135 patients with known or suspected NAFLD. Transfer learning with a deep convolutional neural network (CNN) was applied for quantifying fat fraction and diagnosing fatty liver (≥ 5%) using contemporaneous MRI-PDFF as ground truth. Single and multi-view learning approaches were compared. Class activation mapping generated attention maps to highlight regions important for deep learning-based recognition. The most accurate single view was hepatic veins, with area under the receiver operating characteristic curve (AUC) of 0.86 and Spearman’s rank correlation coefficient of 0.65. A multi-view ensemble of deep-learning models trained for each view separately improved AUC (0.93) and correlation coefficient (0.76). Attention maps highlighted regions known to be used by radiologists in their qualitative assessment, e.g., hepatic vein-parenchyma interface and liver-kidney interface. Machine learning of four liver views can automatically and objectively assess liver fat. Class activation mapping suggests that the CNN focuses on similar features as radiologists. [No. R01DK106419.]