Retinal OCT Images Research Articles

OCTNet: A Modified Multi-Scale Attention Feature Fusion Network with InceptionV3 for Retinal OCT Image Classification

OCTNet: A Modified Multi-Scale Attention Feature Fusion Network with InceptionV3 for Retinal OCT Image Classification

A comparative evaluation of deep learning approaches for ophthalmology

There is a growing number of publicly available ophthalmic imaging datasets and open-source code for Machine Learning algorithms. This allows ophthalmic researchers and practitioners to independently perform various deep-learning tasks. With the advancement in artificial intelligence (AI) and in the field of imaging, the choice of the most appropriate AI architecture for different tasks will vary greatly. The best-performing AI-dataset combination will depend on the specific problem that needs to be solved and the type of data available. The article discusses different machine learning models and deep learning architectures currently used for various ophthalmic imaging modalities and for different machine learning tasks. It also proposes the most appropriate models based on accuracy and other important factors such as training time, the ability to deploy the model on clinical devices/smartphones, heatmaps that enhance the self-explanatory nature of classification decisions, and the ability to train/adapt on small image datasets to determine if further data collection is worthwhile. The article extensively reviews the existing state-of-the-art AI methods focused on useful machine-learning applications for ophthalmology. It estimates their performance and viability through training and evaluating architectures with different public and private image datasets of different modalities, such as full-color retinal images, OCT images, and 3D OCT scans. The article is expected to benefit the readers by enriching their knowledge of artificial intelligence applied to ophthalmology.

PGKD-Net: Prior-guided and Knowledge Diffusive Network for Choroid Segmentation

The thickness of the choroid is considered to be an important indicator of clinical diagnosis. Therefore, accurate choroid segmentation in retinal OCT images is crucial for monitoring various ophthalmic diseases. However, this is still challenging due to the blurry boundaries and interference from other lesions. To address these issues, we propose a novel prior-guided and knowledge diffusive network (PGKD-Net) to fully utilize retinal structural information to highlight choroidal region features and boost segmentation performance. Specifically, it is composed of two parts: a Prior-mask Guided Network (PG-Net) for coarse segmentation and a Knowledge Diffusive Network (KD-Net) for fine segmentation. In addition, we design two novel feature enhancement modules, Multi-Scale Context Aggregation (MSCA) and Multi-Level Feature Fusion (MLFF). The MSCA module captures the long-distance dependencies between features from different receptive fields and improves the model’s ability to learn global context. The MLFF module integrates the cascaded context knowledge learned from PG-Net to benefit fine-level segmentation. Comprehensive experiments are conducted to evaluate the performance of the proposed PGKD-Net. Experimental results show that our proposed method achieves superior segmentation accuracy over other state-of-the-art methods. Our code is made up publicly available at: https://github.com/yzh-hdu/choroid-segmentation.

Improving retinal OCT image classification accuracy using medical pre-training and sample replication methods

Eye disease is a global health issue involving a large population, and it is necessary to develop high-accuracy diagnostic methods. Manual diagnosis has some challenges, and deep learning for automatic diagnosis is becoming a research focus. However, there are two problems in current deep learning used for medical image tasks: deficient network pre-training on ImageNet dataset consisted of natural scene images, which significantly differ from medical images; a discrepancy between sub-network for training and full network for prediction introduced by dropout. In this study, a medical pre-training method and a sample replication method were proposed to address the problems, respectively. The former involves network pre-training on a large-scale medical image dataset called RadImageNet, followed by transfer learning on a retinal OCT (Optical Coherence Tomography) image dataset. The latter uses JS (Jenson's Shannon) divergence between the two predictions yielded with two identical samples via sample replication as an additional loss function to enforce network’s outputs consistency. The two methods were applied on widely used ResNet50, DenseNet121 and InceptionV3 networks, and the effectiveness of using each method and both the two methods to improve the network's performance was demonstrated by experimental results. Compared to those of the corresponding base networks, most the metrics of the networks using the methods were improved for all four sample categories, and the accuracy improvements of 3.76 %, 6.52 % and 8.63 % were obtained for above the networks, respectively. All the networks achieved an accuracy of about 95 % even working with limited target dataset and training resources.

Local Self-Similar Solution of ADMM for Denoising of Retinal OCT Images

Local Self-Similar Solution of ADMM for Denoising of Retinal OCT Images

Lesion-aware attention network for diabetic nephropathy diagnosis with optical coherence tomography images.

For early screening of diabetic nephropathy patients, we propose a deep learning algorithm to screen high-risk patients with diabetic nephropathy from retinal images of diabetic patients. We propose the use of attentional mechanisms to improve the model's focus on lesion-prone regions of retinal OCT images. First, the data is trained using the base network and the Grad-CAM algorithm locates image regions that have a large impact on the model output and generates a rough mask localization map. The mask is used as a auxiliary region to realize the auxiliary attention module. We then inserted the region-guided attention module into the baseline model and trained the CNN model to guide the model to better focus on relevant lesion features. The proposed model improves the recognition of the lesion region. To evaluate the lesion-aware attention network, we trained and tested it using OCT volumetric data collected from 66 patients with diabetic retinal microangiopathy (89 eyes, male = 43, female = 23). There were 45 patients (60 eyes, male=27, female = 18) in DR group and 21 patients (29 eyes, male = 16, female = 5) in DN group. Our proposed model performs even better in disease classification, specifically, the accuracy of the proposed model was 91.68%, the sensitivity was 89.99%, and the specificity was 92.18%. The proposed lesion-aware attention model can provide reliable screening of high-risk patients with diabetic nephropathy.

The effect of optical degradation from cataract using a new Deep Learning optical coherence tomography segmentation algorithm.

To assess the validity of the results of a freely available online Deep Learning segmentation tool and its sensitivity to noise introduced by cataract. The OCT images were collected with a Spectralis SD-OCT (Heidelberg Engineering, Heidelberg, Germany) as part of normal clinical practice. Data were segmented using a freely available online tool called Relayer ( https://www.relayer.online/ ), based on a cross-platform Deep Learning segmentation architecture specifically adapted for retinal OCT images. The segmentations were read into MATLAB (The MathWorks, Natick, MA, USA) and analyzed. There was an excellent agreement between the ETDRS measurements obtained from the two algorithms. Upon visual inspection, the segmentation based on Deep Learning obtained with Relayer appeared more accurate except in one case of apparent good quality image showing interrupted segmentations in some of the B-scans. A freely available online Deep Learning segmentation tool showed good and promising performance in healthy retinas before and after cataract surgery, proving robust to optical degradation of the image from media opacities.

MacD‐Net: An automatic guided‐ensemble approach for macular pathology detection using optical coherence tomography images

AbstractIn this work, a novel guided‐ensemble approach has been proposed for an automated diagnosis of three different types of retinal disorders, such as drusen, choroidal neovascularization (CNV) and diabetic macular edema (DME). These conditions, if left untreated, can lead to vision loss. In the proposed approach, four different pre‐trained CNNs such as MobileNetV1, EfficientNetB3, NASNetMobile and XceptionNet have been fine‐tuned for that purpose and the prediction score obtained by each CNN in an ensemble has been combined based on their validation accuracies. The proposed guided‐ensemble approach has been trained and tested on the public dataset consisting of 84 495 retinal OCT images for training and 1000 unseen images for testing. These images were graded by the experts into four categories, such as CNV, DME, drusen and normal. The proposed approach has outperformed the other methods by obtaining an accuracy of 99.8% in classifying four different classes of retinal OCT images.

Self-Guided Optimization Semi-Supervised Method for Joint Segmentation of Macular Hole and Cystoid Macular Edema in Retinal OCT Images.

Macular hole(MH) and cystoid macular edema(CME) are two common retinal pathologies that cause vision loss. Accurate segmentation of MH and CME in retinal OCT images can greatly aid ophthalmologists to evaluate the relevant diseases. However, it is still challenging as the complicated pathological features of MH and CME in retinal OCT images, such as the diversity of morphologies, low imaging contrast, and blurred boundaries. In addition, the lack of pixel-level annotation data is one of the important factors that hinders the further improvement of segmentation accuracy. Focusing on these challenges, we propose a novel self-guided optimization semi-supervised method termed Semi-SGO for joint segmentation of MH and CME in retinal OCT images. Aiming to improve the model's ability to learn the complicated pathological features of MH and CME, while alleviating the feature learning tendency problem that may be caused by the introduction of skip-connection in U-shaped segmentation architecture, we develop a novel dual decoder dual-task fully convolutional neural network(D3T-FCN). Meanwhile, based on our proposed D3T-FCN, we introduce a knowledge distillation technique to further design a novel semi-supervised segmentation method called Semi-SGO, which can leverage unlabeled data to further improve the segmentation accuracy. Comprehensive experimental results show that our proposed Semi-SGO outperforms other state-of-the-art segmentation networks. Furthermore, we also develop an automatic method for measuring the clinical indicators of MH and CME to validate the clinical significance of our proposed Semi-SGO. The code will be released on Github.

Analysis of Retinal Microstructure in Eyes with Dissociated Optic Nerve Fiber Layer (DONFL) Appearance following Idiopathic Macular Hole Surgery: An Optical Coherence Tomography Study.

(1) Purpose: This study aimed to evaluate morphological changes of the retina in eyes with dissociated optic nerve fiber layer (DONFL) appearance following internal limiting membrane (ILM) peeling for full-thickness idiopathic macular hole (IMH) on spectral-domain optical coherence tomography (SD-OCT). (2) Methods: We retrospectively analyzed 39 eyes of 39 patients with type 1 macular hole closure after a vitrectomy with ILM peeling procedure at a six-month minimum postoperative follow-up. The retinal thickness maps and cross-sectional OCT images were obtained from a clinical OCT device. The cross-sectional area of the retinal nerve fiber layer (RNFL) on cross-sectional OCT images was manually measured by ImageJ software. (3) Results: The inner retinal layers (IRLs) thickness thinned down much more in the temporal quadrant than in nasal quadrants at 2 and 6 months postoperatively (p < 0.001). However, the cross-sectional area of the RNFL did not change significantly at 2 and 6 months postoperatively (p > 0.05) when compared to preoperative data. In addition, the thinning of the IRL did not correlate with the best-corrected visual acuity (BCVA) at 6 months postoperatively. (4) Conclusions: The thickness of the IRL decreased in eyes with a DONFL appearance after ILM peeling for IMH. The thickness of the IRL decreased more in the temporal retina than in the nasal retina, but the change did not affect BCVA during the 6 months after surgery.

Unsupervised Denoising of Retinal OCT Images Based on Deep Learning

Unsupervised Denoising of Retinal OCT Images Based on Deep Learning

TSSK-Net: Weakly supervised biomarker localization and segmentation with image-level annotation in retinal OCT images

The localization and segmentation of biomarkers in OCT images are critical steps in retina-related disease diagnosis. Although fully supervised deep learning models can segment pathological regions, their performance relies on labor-intensive pixel-level annotations. Compared with dense pixel-level annotation, image-level annotation can reduce the burden of manual annotation. Existing methods for image-level annotation are usually based on class activation maps (CAM). However, current methods still suffer from model collapse, training instability, and anatomical mismatch due to the considerable variation in retinal biomarkers' shape, texture, and size. This paper proposes a novel weakly supervised biomarkers localization and segmentation method, requiring only image-level annotations. The technique is a Teacher-Student network with joint Self-supervised contrastive learning and Knowledge distillation-based anomaly localization, namely TSSK-Net. Specifically, we treat retinal biomarker regions as abnormal regions distinct from normal regions. First, we propose a novel pre-training strategy based on supervised contrastive learning that encourages the model to learn the anatomical structure of normal OCT images. Second, we design a fine-tuning module and propose a novel hybrid network structure. The network includes supervised contrastive loss for feature learning and cross-entropy loss for classification learning. To further improve the performance, we propose an efficient strategy to combine these two losses to preserve the anatomical structure and enhance the encoding representation of features. Finally, we design a knowledge distillation-based anomaly segmentation method that is effectively combined with the previous model to alleviate the challenge of insufficient supervision. Experimental results on a local dataset and a public dataset demonstrated the effectiveness of our proposed method. Our proposed method can effectively reduce the annotation burden of ophthalmologists in OCT images.

Spatial–contextual variational autoencoder with attention correction for anomaly detection in retinal OCT images

Anomaly detection refers to leveraging only normal data to train a model for identifying unseen abnormal cases, which is extensively studied in various fields. Most previous methods are based on reconstruction models, and use anomaly score calculated by the reconstruction error as the metric to tackle anomaly detection. However, these methods just employ single constraint on latent space to construct reconstruction model, resulting in limited performance in anomaly detection. To address this problem, we propose a Spatial–Contextual Variational Autoencoder with Attention Correction for anomaly detection in retinal OCT images. Specifically, we first propose a self-supervised segmentation network to extract retinal regions, which can effectively eliminate interference of background regions. Next, by introducing both multi-dimensional and one-dimensional latent space, our proposed framework can then learn the spatial and contextual manifolds of normal images, which is conducive to enlarging the difference between reconstruction errors of normal images and those of abnormal ones. Furthermore, an ablation-based method is proposed to localize anomalous regions by computing the importance of feature maps, which is used to correct anomaly score calculated by reconstruction error. Finally, a novel anomaly score is constructed to separate the abnormal images from the normal ones. Extensive experiments on two retinal OCT datasets are conducted to evaluate our proposed method, and the experimental results demonstrate the effectiveness of our approach.

Stochastic Differential Equations for Automatic Quality Control of Retinal Optical Coherence Tomography images.

Optical coherence tomography is widely used to provide high resolution images from retina. During data acquisition, several artifacts may be associated with OCT images which clearly remove information of retinal layers and degrade the quality of images. Manual assessment of the acquired OCT images is hard and time consuming. Therefore, an automatic quality control step is necessary to detect poor images for next decisions of eliminating them and even re-scanning. In this study, a novel automatic quality control methodology is proposed for early assessment of the OCT images quality by employing stochastic differential equations (SDE). In this method α-stable nature of OCT images is represented by applying a fractional Laplacian filter and parameters of the obtained α-stable are fed to an SVM to automatically detect high quality vs poor quality images. The simulation results on a large dataset of normal and abnormal OCT images show that proposed method has outstanding performance in detection of poor vs high quality images. The methodology is applicable to the image quality assessment of other OCT scanning devices as well. Clinical Relevance- Automatic quality control assessment of retinal OCT images provides reliable data for diagnosis of retinal and systematic diseases in clinical applications.

A comparative approach for classifying retinal OCT images based on deep learning framework

Convolutional Networks are category of deep optimizing networks used to interpret images in Deep Learning concepts. Image recognition and medical image analysis are two areas where they are useful. The increasing scale of clinical feature spaces is raising a significant obstacle, creating issues with extensive database management, and afterward compiling those repositories for retrieval and storage, that could only be addressed using content based medical image retrieval systems. The objective of this paper is to demonstrate a deep CNN architecture for retrieving research and clinical images quickly and efficiently for identifying multi-class retinal disease objects. To train the network, the datasets used are inter-modal and divided into 4 groups. The transfer learning method is used for the multi-classification of retinal images. Another augmentation technique is used for comparing the accuracy, precision, and evaluation metrics with the transfer learning method. The accuracy of 97.1%, with a recall of 97.2%, and a precision of 97.0% was achieved in research that is higher when compared with the previous methods that were deployed. With the augmentation technique, it achieved an accuracy of 94.0% with a 94.6% precision and a recall of 95.1% for the testing data which suggests that decreasing the size of data did not impact the accuracy of the model. The proposed model helps diagnose various categories of medical images for the development of a comprehensive system that can work better than the human experts and help to detect and diagnose various diseases in the medical and clinical fields.

A Novel N-Gram-Based Image Classification Model and Its Applications in Diagnosing Thyroid Nodule and Retinal OCT Images.

Imbalanced classes and dimensional disasters are critical challenges in medical image classification. As a classical machine learning model, the n-gram model has shown excellent performance in addressing this issue in text classification. In this study, we proposed an algorithm to classify medical images by extracting their n-gram semantic features. This algorithm first converts an image classification problem to a text classification problem by building an n-gram corpus for an image. After that, the algorithm was based on the n-gram model to classify images. The algorithm was evaluated by two independent public datasets. The first experiment is to diagnose benign and malignant thyroid nodules. The best area under the curve (AUC) is 0.989. The second experiment is to diagnose the type of fundus lesion. The best result is that it correctly identified 86.667% of patients with dry age-related macular degeneration (AMD), 93.333% of patients with diabetic macular edema (DME), and 93.333% of normal individuals.

Proxy-Bridged Image Reconstruction Network for Anomaly Detection in Medical Images.

Anomaly detection in medical images refers to the identification of abnormal images with only normal images in the training set. Most existing methods solve this problem with a self-reconstruction framework, which tends to learn an identity mapping and reduces the sensitivity to anomalies. To mitigate this problem, in this paper, we propose a novel Proxy-bridged Image Reconstruction Network (ProxyAno) for anomaly detection in medical images. Specifically, we use an intermediate proxy to bridge the input image and the reconstructed image. We study different proxy types, and we find that the superpixel-image (SI) is the best one. We set all pixels' intensities within each superpixel as their average intensity, and denote this image as SI. The proposed ProxyAno consists of two modules, a Proxy Extraction Module and an Image Reconstruction Module. In the Proxy Extraction Module, a memory is introduced to memorize the feature correspondence for normal image to its corresponding SI, while the memorized correspondence does not apply to the abnormal images, which leads to the information loss for abnormal image and facilitates the anomaly detection. In the Image Reconstruction Module, we map an SI to its reconstructed image. Further, we crop a patch from the image and paste it on the normal SI to mimic the anomalies, and enforce the network to reconstruct the normal image even with the pseudo abnormal SI. In this way, our network enlarges the reconstruction error for anomalies. Extensive experiments on brain MR images, retinal OCT images and retinal fundus images verify the effectiveness of our method for both image-level and pixel-level anomaly detection.

MsTGANet: Automatic Drusen Segmentation From Retinal OCT Images.

Drusen is considered as the landmark for diagnosis of AMD and important risk factor for the development of AMD. Therefore, accurate segmentation of drusen in retinal OCT images is crucial for early diagnosis of AMD. However, drusen segmentation in retinal OCT images is still very challenging due to the large variations in size and shape of drusen, blurred boundaries, and speckle noise interference. Moreover, the lack of OCT dataset with pixel-level annotation is also a vital factor hindering the improvement of drusen segmentation accuracy. To solve these problems, a novel multi-scale transformer global attention network (MsTGANet) is proposed for drusen segmentation in retinal OCT images. In MsTGANet, which is based on U-Shape architecture, a novel multi-scale transformer non-local (MsTNL) module is designed and inserted into the top of encoder path, aiming at capturing multi-scale non-local features with long-range dependencies from different layers of encoder. Meanwhile, a novel multi-semantic global channel and spatial joint attention module (MsGCS) between encoder and decoder is proposed to guide the model to fuse different semantic features, thereby improving the model's ability to learn multi-semantic global contextual information. Furthermore, to alleviate the shortage of labeled data, we propose a novel semi-supervised version of MsTGANet (Semi-MsTGANet) based on pseudo-labeled data augmentation strategy, which can leverage a large amount of unlabeled data to further improve the segmentation performance. Finally, comprehensive experiments are conducted to evaluate the performance of the proposed MsTGANet and Semi-MsTGANet. The experimental results show that our proposed methods achieve better segmentation accuracy than other state-of-the-art CNN-based methods.

Circumpapillary Retinal Nerve Fiber Layer OCT Imaging in a Parkinson's Disease Cohort-A Multidisciplinary Approach in a Clinical Research Hospital.

(1) Background: The purpose of this paper is to report the data of the first study in a Clinical Research Hospital, in the Transylvania region, focusing on the Spectral Domain Optical Coherence Tomography (SD-OCT) measurements in the early stages of Parkinson’s disease (PD), and to compare the results with age-matched healthy controls. (2) Methods: This study assessed the circumpapillary retinal nerve fiber layer (cpRNFL) SD-OCT measurements (Heidelberg Spectralis, Heidelberg Engineering, Germany) of two study groups: patients suffering from PD (Hoehn−Yahr stages 1–3) and healthy controls. Secondary objectives were to investigate the reported visual symptoms by evaluating the color vision, contrast sensitivity, and the central visual defects for macular disease using standardized charts. Subjects with prior history of ophthalmologic diseases, advanced stages of PD (Hoehn−Yahr stages 4–5), or with psychiatric conditions were not included in this study. The same team of neurologists and ophthalmologists evaluated all individuals in order to have comparable data and to eliminate inter-examiner differences. All subjects were recruited from the same Clinical Research Hospital in the Transylvania region, Romania. (3) Results: 72% of the PD patients (n = 17) in this study reported visual symptoms. In respect to the ophthalmologic chart evaluation for PD patients, the most frequent disturbances were identified in the Ishihara color perception testing (33%). The regression analysis showed significant results for the Ishihara testing in relation to the cpRNFL thinning in the temporal retinal sectors for both eyes. cpRNFL thinning was predominantly contralateral to the parkinsonism (p = 0.001). The temporal and global values of the cpRNFL were significantly lower in all PD patients < 70 years old, compared to the age-matched healthy controls. (4) Conclusions: Specific patterns of cpRNFL thinning were found in the PD subjects younger than 70 years. A multidisciplinary approach is essential for a complete evaluation of PD patients.

Automatic segmentation of retinal edema in optical coherence tomography based on deep neural networks

A multi-task network framework based on convolutional neural networks improvements is proposed for automatic segmentation and classification in three different lesion areas of retinal OCT images. We propose a dilated convolutions module to capture fine-grained semantic information and introduce dense skip connections to integrate features of multiple scales and different semantic levels. Furthermore, we use a joint loss function that balances the relative size of the data labels and the improved generalization of the network. The validity and reliability of our proposed method are trained and verified on OCT datasets. Extensive experimental results demonstrate that the average Dice coefficient of the three lesion areas is 2.61% higher than the most commonly used U-net model in the field of medical image segmentation and reaches 0.81501.