Let’s Find Fluorescein: Cross-Modal Dual Attention Learning For Fluorescein Leakage Segmentation In Fundus Fluorescein Angiography

Abstract

Automatic segmentation of fluorescein leakage in fundus fluorescein angiography images is important in the clinical diagnosis of advanced diabetic retinopathy. Despite the recent success of deep-learning-based models in improving medical image segmentation, segmentation of fluorescein leakage has been ignored owing to (1) a lack of publicly available data with sufficient annotations for training a segmentation network and (2) incapability of supervised models to accurately localize fluorescein leakage at different imaging angles. To address these issues, we studied the automatic segmentation of fluorescein leakage in fundus fluorescein angiography images and devised a method involving (1) a cross-modal learning framework for fluorescein leakage segmentation using both image and text data, (2) a dual attention learning module for identifying important linguistic and visual features, and (3) fluorescein-related-keyword classification for identifying meaningful textual expressions pertaining to the location and type of fluorescein leakage. We demonstrate the effectiveness of the proposed method for an in-house fundus fluorescein angiography image data set.