FLeak-Seg: Automated Fundus Fluorescein Leakage Segmentation via Cross-Modal Attention Learning

Abstract

Despite the recent success of deep learning-based models for medical image segmentation and the importance of automated fluorescein leakage segmentation for the diagnosis of advanced diabetic retinopathy, segmentation of fluorescein leakage has been neglected because 1) there are no publicly available databases with sufficient annotations to train segmentation models and 2) supervised models struggle to accurately distinguish between different types of fluorescein leakage and localize leakages at different imaging angles. To tackle these challenges, this work presents FLeak-Seg, a cross-modal dual attention learning method to jointly capture visual and language information, for end-to-end fluorescein leakage segmentation in fundus fluorescein angiography. Specifically, both image and text data are used as input, where visual and linguistic features are captured by a cross-modal attention learning module to compensate for the lack of annotations. A keyword classification module is also employed to identify meaningful expressions related to the type and location of fluorescein leakages to further facilitate the segmentation. Experimental results obtained in an in-house fundus fluorescein angiography database demonstrate the superiority of our method. We show how erroneous segmentation masks can be improved using FLeak-Seg, its advantages in the context of limited samples, and its behavior on segmenting different types of fluorescein leakages.