Multimodal deep transfer learning to predict retinal vein occlusion macular edema recurrence after anti-VEGF therapy

Abstract

PurposeTo develop a multimodal deep transfer learning (DTL) fusion model using optical coherence tomography angiography (OCTA) images to predict the recurrence of retinal vein occlusion (RVO) and macular edema (ME) after three consecutive anti-VEGF therapies. MethodsThis retrospective cross-sectional study consisted of 2800 B-scan OCTA macular images collected from 140 patients with RVO-ME. The central macular thickness (CMT) > 250 μm was used as a criterion for recurrence in the three-month follow-up after three injections of anti-VEGF therapy. The qualified OCTA image preprocessing and the lesion area segmentation were performed by senior ophthalmologists. We developed and validated the clinical, DTL, and multimodal fusion models based on clinical and extracted OCTA imaging features. The performance of the models and experts predictions were evaluated using several performance metrics, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity. ResultsThe DTL models exhibited higher prediction efficacy than the clinical models and experts’ predictions. Among the DTL models, the Vgg19 performed better than that of the other models, with an AUC of 0.968 (95 % CI, 0.943–0.994), accuracy of 0.913, sensitivity of 0.922, and specificity of 0.902 in the validation cohort. Moreover, the fusion Vgg19 model showed the highest prediction efficacy among all the models, with an AUC of 0.972 (95 % CI, 0.946–0.997), accuracy of 0.935, sensitivity of 0.935, and specificity of 0.934 in the validation cohort. ConclusionsMultimodal fusion DTL models showed robust performance in predicting RVO-ME recurrence and may be applied to assist clinicians in determining patients’ follow-up time after anti-VEGF therapy.