A spatiotemporal convolution recurrent neural network for pixel-level peripapillary atrophy prediction using sequential fundus images

Abstract

The progression of peripapillary atrophy (PPA) is closely associated with the development of retinal diseases such as myopia and glaucoma. PPA prediction employing longitudinal images to obtain its progress trend can facilitate personalized treatment. Although existing studies have attempted to predict the persistence of PPA, such studies cannot provide quantitative measurement for personalized treatment. In this paper, we propose a spatiotemporal framework for pixel-level PPA prediction using sequential fundus images, including feature extractor, temporal memory, and spatiotemporal prediction modules. To take advantage of historical information, a temporal memory module is used, integrating current and prior features to build sequential data of features. To further enhance the prediction performance, the recurrent neural network states in a spatiotemporal prediction module transmit between different layers, enabling high-level states to guide the learning of low-level states. To handle missing data in clinical follow-up data, we use the predicted output of the spatiotemporal prediction module to substitute the missing data, and the scheduled-sampling strategy is employed in training. Extensive experiments conducted using a clinical dataset demonstrate that our proposed method achieves a satisfactory performance compared with the start-of-the-art models. The proposed approach can be applied using clinical data to obtain various quantitative indicators for personalized treatment and prevention of retinal disease.