Lesions In Fundus Images Research Articles

Automated Machine Learning versus Expert-Designed Models in Ocular Toxoplasmosis: Detection and Lesion Localization Using Fundus Images

ABSTRACT Purpose Automated machine learning (AutoML) allows clinicians without coding experience to build their own deep learning (DL) models. This study assesses the performance of AutoML in detecting and localizing ocular toxoplasmosis (OT) lesions in fundus images and compares it to expert-designed models. Methods Ophthalmology trainees without coding experience designed AutoML models using 304 labelled fundus images. We designed a binary model to differentiate OT from normal and an object detection model to visually identify OT lesions. Results The AutoML model had an area under the precision-recall curve (AuPRC) of 0.945, sensitivity of 100%, specificity of 83% and accuracy of 93.5% (vs. 94%, 86% and 91% for the bespoke models). The AutoML object detection model had an AuPRC of 0.600 with a precision of 93.3% and recall of 56%. Using a diversified external validation dataset, our model correctly labeled 15 normal fundus images (100%) and 15 OT fundus images (100%), with a mean confidence score of 0.965 and 0.963, respectively. Conclusion AutoML models created by ophthalmologists without coding experience were comparable or better than expert-designed bespoke models trained on the same dataset. By creatively using AutoML to identify OT lesions on fundus images, our approach brings the whole spectrum of DL model design into the hands of clinicians.

GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathy.

Early-stage diabetic retinopathy (DR) presents challenges in clinical diagnosis due to inconspicuous and minute microaneurysms (MAs), resulting in limited research in this area. Additionally, the potential of emerging foundation models, such as the segment anything model (SAM), in medical scenarios remains rarely explored. In this work, we propose a human-in-the-loop, label-free early DR diagnosis framework called GlanceSeg, based on SAM. GlanceSeg enables real-time segmentation of MA lesions as ophthalmologists review fundus images. Our human-in-the-loop framework integrates the ophthalmologist's gaze maps, allowing for rough localization of minute lesions in fundus images. Subsequently, a saliency map is generated based on the located region of interest, which provides prompt points to assist the foundation model in efficiently segmenting MAs. Finally, a domain knowledge filtering (DKF) module refines the segmentation of minute lesions. We conducted experiments on two newly-built public datasets, i.e., IDRiD and Retinal-Lesions, and validated the feasibility and superiority of GlanceSeg through visualized illustrations and quantitative measures. Additionally, we demonstrated that GlanceSeg improves annotation efficiency for clinicians and further enhances segmentation performance through fine-tuning using annotations. The clinician-friendly GlanceSeg is able to segment small lesions in real-time, showing potential for clinical applications.

Multi-Scale Class Attention Network for Diabetes Retinopathy Grading

Article Multi-Scale Class Attention Network for Diabetes Retinopathy Grading Hongyu Chen 1, Ronghua Wu 1, Chen Tao 1, Wenjing Xu 1, Hongzhe Liu 3, Cheng Xu 3, and Muwei Jian 1,2,* 1 School of Information Science and Technology, Linyi University, 276000, China 2 School of Computer Science and Technology, Shandong University of Finance and Economics, 250014, China 3 Beijing Key Laboratory of Information Service Engineering, 100101, Beijing, China * Correspondence: jianmuweihk@163.com Received: 1 June 2023 Accepted: 21 July 2023 Abstract: Diabetes retinopathy (DR) is a universal eye disease, which brings irreversible blindness risks to patients in severe cases. Due to the scarcity of professional ophthalmologists, it has become increasingly important to develop computer-aided diagnostic systems for DR grading diagnosis. However, the current mainstream deep learning methods face challenges in accurately classifying the severity of DR, making it difficult for them to provide a reliable reference for clinicians. To tackle this problem, we propose two novel modules to improve the accuracy of DR classification. Specifically, we design a multi-scale feature extraction module to capture tiny lesions in fundus images and differentiate similar lesions simultaneously. In addition, we develop a class attention module to alleviate the adverse impact of intra-class similarity on DR grading. Experimental results show that our proposed modules attain significant performance improvement on the APTOS2019 blind detection dataset, with accuracy and quadratic weighted Kappa metrics achieving 95.98% and 97.12%, respectively.

Multi-Scale Feature Pyramid for Detection of Red Lesions in Fundus Images

Diabetic retinopathy (DR) is increasing rapidly around the world, but there is a shortage of experienced ophthalmologists. Therefore, computer-based diagnosis of the fundus images is essential to screening of referable DR. Automated detection of red lesions is very important for screening of DR. This paper deals with a novel method for automatic detection of red lesion. The main contribution is developing a deep learning based detection framework to handle severe class imbalance and imbalance in sizes of red lesions. The multi-scale features are extracted using the feature pyramid network. A pyramid of features is generated with strong semantics. The proposed network is end-to-end trainable in image level with several scales and works for a wide range of red lesions with acceptable performance. Sensitivity of the proposed method is 0.76 with six false-positive per image on test set of publicly available DIARECTDB1 database and outperforms state-of-the-art approaches. A potential benefit with deep learning based detection framework could be used in screening programs of referable DR.

Segmentation of hard exudate lesions in color fundus image using two-stage CNN-based methods

The presence of hard exudate (EX) lesions is an early clinical symptom of Diabetic Retinopathy (DR); its accurate segmentation is essential for diagnosis and treatment. Automatic segmentation of EX lesions is challenging because they have a wide range of sizes, shapes, and brightness, and can be confused with soft exudate lesions. In this paper, we present and assess an efficient segmentation method for interactively segmenting EX lesions in fundus images from DR patients. Our approach consisted of two main stages: (1) the first stage generates an automatic segmentation using a CNN-based method. (2) If the obtained segmentation is suboptimal, the second stage is used to refine each inaccurately segmented region using a CNN-based interactive segmentation method. To train and evaluate method performance, we used two public datasets (IDRiD and DDR) and two local datasets from two medical centers (EHos and EWT) containing a wide range of EX lesions. Several state-of-the art CNN-based models and the proposed method were trained and evaluated on IDRiD, DDR and EHos datasets using the area under the precision–recall curve (AUPR), the dice similarity coefficient (Dice) and the intersection over union (IoU). In addition, EWT dataset was used to evaluate the methods without retraining the CNN models. The evaluation results using IDRiD dataset showed that the proposed method achieved AUPR, mean Dice and mean IoU scores of 0.893, 76.6% and 62.4%, respectively, which are comparable to the state-of-the-art method on EX segmentation. When evaluated on the DDR dataset with large portion of small EX lesions, the proposed method achieved AUPR, mean Dice and mean IoU scores of 0.693, 60.8% and 45.5%, yielding state-of-the-art performance. Furthermore, with the main evaluation metric being AUPR, the evaluation results on two datasets, EHos and ETW, with scores of 0.689 and 0.667 demonstrated superiority of the proposed method over the other segmentation methods. The proposed two-stage CNN-based segmentation method is an useful approach for accurately segmenting EX lesions in color fundus images, enabling lesion quantification to aid experts in treating patients with DR. The source code, the trained CNN models and the data are publicly available at: https://github.com/dvquang2000/Interactive_HardExudates_Segmentation.

Individualized Statistical Modeling of Lesions in Fundus Images for Anomaly Detection.

Anomaly detection in fundus images remains challenging due to the fact that fundus images often contain diverse types of lesions with various properties in locations, sizes, shapes, and colors. Current methods achieve anomaly detection mainly through reconstructing or separating the fundus image background from a fundus image under the guidance of a set of normal fundus images. The reconstruction methods, however, ignore the constraint from lesions. The separation methods primarily model the diverse lesions with pixel-based independent and identical distributed (i.i.d.) properties, neglecting the individualized variations of different types of lesions and their structural properties. And hence, these methods may have difficulty to well distinguish lesions from fundus image backgrounds especially with the normal personalized variations (NPV). To address these challenges, we propose a patch-based non-i.i.d. mixture of Gaussian (MoG) to model diverse lesions for adapting to their statistical distribution variations in different fundus images and their patch-like structural properties. Further, we particularly introduce the weighted Schatten p-norm as the metric of low-rank decomposition for enhancing the accuracy of the learned fundus image backgrounds and reducing false-positives caused by NPV. With the individualized modeling of the diverse lesions and the background learning, fundus image backgrounds and NPV are finely learned and subsequently distinguished from diverse lesions, to ultimately improve the anomaly detection. The proposed method is evaluated on two real-world databases and one artificial database, outperforming the state-of-the-art methods.

Classification and Segmentation of Diabetic Retinopathy: A Systemic Review

Diabetic retinopathy (DR) is a major reason of blindness around the world. The ophthalmologist manually analyzes the morphological alterations in veins of retina, and lesions in fundus images that is a time-taking, costly, and challenging procedure. It can be made easier with the assistance of computer aided diagnostic system (CADs) that are utilized for the diagnosis of DR lesions. Artificial intelligence (AI) based machine/deep learning methods performs vital role to increase the performance of the detection process, especially in the context of analyzing medical fundus images. In this paper, several current approaches of preprocessing, segmentation, feature extraction/selection, and classification are discussed for the detection of DR lesions. This survey paper also includes a detailed description of DR datasets that are accessible by the researcher for the identification of DR lesions. The existing methods limitations and challenges are also addressed, which will assist invoice researchers to start their work in this domain.

Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology

Fundus image retrieval can help ophthalmologists make evidence-based medico-decision by providing similar cases. Its basic task is to learn highly discriminative visual descriptors from image space, in which lesion features are the main differentiating clue. Lesions in fundus images appear small in size, similar in textures, and scatter around vessels, such as microaneurysms and hemorrhages. Hence, although a single small lesion has a saliently visual manifestation, its discriminative information is hard to reserve in the last image descriptors. For fundus images, the optic disc of the left and right eyes are symmetric, and the macular area lies in the central axis from the vertical view. Based on such spatial structure and lesion characteristics, we present a novel deep metric learning framework equipped with mirror attention to enhance the discriminative features of small and scattering lesions and encode them into image descriptors. The mirror attention can give lesions high attention scores by capturing spatial dependency of vertical and horizontal views, especially the relations between lesions and vessels. Based on the mirror attention, we further propose a new fine triplet loss to confine distances of positive pairs by exploiting the learned relevant degrees of positive pairs in a self-supervised manner. The fine triplet loss can help detect the subtle differences of positive pairs to improve the ranking performance of hit items. To demonstrate the effectiveness of improving retrieval performance, we conduct comprehensive experiments on the largest fundus dataset of diabetic retinopathy (DR) detection and achieve the best precision compared to counterparts. The experiments show that our method produces significant performance improvements for fundus image retrieval, especially the ranking quality of DR grades containing microaneurysms and hemorrhages. Our proposed mirror attention can be applied to off-the-shelf backbones and trained efficiently in an end-to-end manner for other medical images to obtain highly discriminative image descriptors.

Multi-Lesion Segmentation of Diabetic Retinopathy Using Deep Learning

Abstract: Diabetic Retinopathy (DR) and Diabetic Macular Edema (DME) are the two major complications of diabetes and have a significant impact on working individuals of the world population. DR doesn’t give any early symptoms. Therefore, it is important to diagnose DR at an early stage. The two above mentioned diseases usually depend on the presence and areas of lesions in fundus images. The four main related lesions include soft exudates, hard exudates,microaneurysms, and haemorrhages. Since lesions in retinal fundus images are a pivotal indicator of DR, analyzing retinal fundus images is the most popular method for DR screening. The examination of fundus images is time-consuming and small lesions are hard to observe. Therefore, adopting deep learning techniques for lesion segmentation is of great importance. In this project, we use one of the deep learning techniques called U-Net, which is a variant of Convolutional Neural Networks (CNN) for multiple lesion segmentation.

An Efficient Hierarchical Optic Disc and Cup Segmentation Network Combined with Multi-task Learning and Adversarial Learning.

Automatic and accurate segmentation of optic disc (OD) and optic cup (OC) in fundus images is a fundamental task in computer-aided ocular pathologies diagnosis. The complex structures, such as blood vessels and macular region, and the existence of lesions in fundus images bring great challenges to the segmentation task. Recently, the convolutional neural network-based methods have exhibited its potential in fundus image analysis. In this paper, we propose a cascaded two-stage network architecture for robust and accurate OD and OC segmentation in fundus images. In the first stage, the U-Net like framework with an improved attention mechanism and focal loss is proposed to detect accurate and reliable OD location from the full-scale resolution fundus images. Based on the outputs of the first stage, a refined segmentation network in the second stage that integrates multi-task framework and adversarial learning is further designed for OD and OC segmentation separately. The multi-task framework is conducted to predict the OD and OC masks by simultaneously estimating contours and distance maps as auxiliary tasks, which can guarantee the smoothness and shape of object in segmentation predictions. The adversarial learning technique is introduced to encourage the segmentation network to produce an output that is consistent with the true labels in space and shape distribution. We evaluate the performance of our method using two public retinal fundus image datasets (RIM-ONE-r3 and REFUGE). Extensive ablation studies and comparison experiments with existing methods demonstrate that our approach can produce competitive performance compared with state-of-the-art methods.

Diabetic retinopathy detection and classification using capsule networks

Nowadays, diabetic retinopathy is a prominent reason for blindness among the people who suffer from diabetes. Early and timely detection of this problem is critical for a good prognosis. An automated system for this purpose contains several phases like identification and classification of lesions in fundus images. Machine learning techniques based on manual extraction of features and automatic extraction of features with convolution neural network have been presented for diabetic retinopathy detection. The recent developments like capsule networks in deep learning and their significant success over traditional machine learning methods for a variety of applications inspired the researchers to apply them for diabetic retinopathy diagnosis. In this paper, a reformed capsule network is developed for the detection and classification of diabetic retinopathy. Using the convolution and primary capsule layer, the features are extracted from the fundus images and then using the class capsule layer and softmax layer the probability that the image belongs to a specific class is estimated. The efficiency of the proposed reformed network is validated concerning four performance measures by considering the Messidor dataset. The constructed capsule network attains an accuracy of 97.98%, 97.65%, 97.65%, and 98.64% on the healthy retina, stage 1, stage 2, and stage 3 fundus images.

Red Lesions Identification of Retina for Diabetic Patients

The automatic identification of retinal lesions in fundus images is crucial. In this paper, the approach used for identification of microaneurysms and hemorrhages in fundus images is appropriate. By using this technique, the features of changing shape for the identification of red lesions in the fundus images are effective in which segmentation is not necessary. While image flooding, shape features are evolved and characterized between red lesions and vessel segments.

A novel lesion detection algorithm based on multi-scale input convolutional neural network model for diabetic retinopathy

Objective To propose a multi-scale convolutional neural network (CNN) based lesions detection method of fundus image, and evaluate its application in diabetic retinopathy (DR) assisted diagnosis. Methods A multi-scale CNN based on lesions detection method of fundus image was proposed.Compared with the existing detection methods, the problem of poor robustness based on threshold segmentation and morphological segmentation was overcome.The idea of multi-scale grids detection without relying on manual pixel-by-pixel labeling was adopted in this algorithm, and the detection performance of small lesions was significantly improved.In addition, multiple DR lesions with high accuracy could be detected by the proposed loss function under the condition of weak labels and small data sets. Results At the level of lesions, the sensitivity and specificity of hard exudation lesions detection were 92.17% and 97.17%, respectively.Compared with single-scale method, the sensitivity and accuracy of multi-scale method proposed in this paper increased by 7.41% and 5.02%, respectively, and compared with other algorithm using the same public dataset IDRiD, the specificity of this algorithm increased by 55.82%.This method could effectively detect the lesions in fundus images, and could give the basic range of the lesions.The average detection time of fundus images with a large number of lesions was 1.59 seconds. Conclusions The DR lesions in the fundus image can be quickly and reliably identified, the location information of the lesions can be marked, and the influence of subjective factors can be reduced by using this algorithm, and it can be used to assist the clinician to conduct more effectively. Key words: Artificial intelligence; Diabetic retinopathy/diagnosis; Fundus color photograph; Multi-scale convolutional neural network

Deep learning based computer-aided diagnosis systems for diabetic retinopathy: A survey.

Diabetic retinopathy (DR) results in vision loss if not treated early. A computer-aided diagnosis (CAD) system based on retinal fundus images is an efficient and effective method for early DR diagnosis and assisting experts. A computer-aided diagnosis (CAD) system involves various stages like detection, segmentation and classification of lesions in fundus images. Many traditional machine-learning (ML) techniques based on hand-engineered features have been introduced. The recent emergence of deep learning (DL) and its decisive victory over traditional ML methods for various applications motivated the researchers to employ it for DR diagnosis, and many deep-learning-based methods have been introduced. In this paper, we review these methods, highlighting their pros and cons. In addition, we point out the challenges to be addressed in designing and learning about efficient, effective and robust deep-learning algorithms for various problems in DR diagnosis and draw attention to directions for future research.

L-Seg: An end-to-end unified framework for multi-lesion segmentation of fundus images

Diabetic retinopathy and diabetic macular edema are the two leading causes for blindness in working-age people, and the quantitative and qualitative diagnosis of these two diseases usually depends on the presence and areas of lesions in fundus images. The main related lesions include soft exudates, hard exudates, microaneurysms, and haemorrhages. However, segmentation of these four kinds of lesions is difficult due to their uncertainty in size, contrast, and high interclass similarity. Therefore, we aim to design a multi-lesion segmentation model. We have designed the first small object segmentation network (L-Seg) that can segment the four kinds of lesions simultaneously. Taking into account that small lesion regions could not response at high level of network, we propose a multi-scale feature fusion method to handle this problem. In addition, when considering the cases of both class-imbalance and loss-imbalance problems, we propose a multi-channel bin loss. We have evaluated L-Seg on three fundus datasets including two publicly available datasets - IDRiD and e-ophtha and one private dataset - DDR. Extensive experiments have demonstrated that L-Seg achieves better performance in small lesion segmentation than other deep learning models and traditional methods. Specially, the mAUC score of L-Seg is over 16.8%, 1.51% and 3.11% higher than that of DeepLab v3+ on IDRiD, e-ophtha and DDR datasets, respectively. Moreover, our framework shows competitive performance compared with top-3 teams in IDRiD challenge. The source code of L-Seg is available at: https://github.com/guomugong/L-Seg.

Red Lesion Detection Using Dynamic Shape Features for Diabetic Retinopathy Screening.

The development of an automatic telemedicine system for computer-aided screening and grading of diabetic retinopathy depends on reliable detection of retinal lesions in fundus images. In this paper, a novel method for automatic detection of both microaneurysms and hemorrhages in color fundus images is described and validated. The main contribution is a new set of shape features, called Dynamic Shape Features, that do not require precise segmentation of the regions to be classified. These features represent the evolution of the shape during image flooding and allow to discriminate between lesions and vessel segments. The method is validated per-lesion and per-image using six databases, four of which are publicly available. It proves to be robust with respect to variability in image resolution, quality and acquisition system. On the Retinopathy Online Challenge's database, the method achieves a FROC score of 0.420 which ranks it fourth. On the Messidor database, when detecting images with diabetic retinopathy, the proposed method achieves an area under the ROC curve of 0.899, comparable to the score of human experts, and it outperforms state-of-the-art approaches.

Ccl2, Cx3cr1 and Ccl2/Cx3cr1 chemokine deficiencies are not sufficient to cause age-related retinal degeneration

Monocytes, macrophages, dendritic cells and microglia play critical roles in the local immune response to acute and chronic tissue injury and have been implicated in the pathogenesis of age-related macular degeneration. Defects in Ccl2-Ccr2 and Cx3cl1-Cx3cr1 chemokine signalling cause enhanced accumulation of bloated subretinal microglia/macrophages in senescent mice and this phenomenon is reported to result in the acceleration of age-related retinal degeneration. The purpose of this study was to determine whether defects in CCL2-CCR2 and CX3CL1-CX3CR1 signalling pathways, alone or in combination, cause age-dependent retinal degeneration. We tested whether three chemokine knockout mouse lines, Ccl2−/−, Cx3cr1−/− and Ccl2−/−/Cx3cr1−/−, in comparison to age-matched C57Bl/6 control mice show differences in subretinal macrophage accumulation and loss of adjacent photoreceptor cells at 12–14 months of age. All mouse lines are derived from common parental strains and do not carry the homozygous rd8 mutation in the Crb1 gene that has been a major confounding factor in previous reports. We quantified subretinal macrophages by counting autofluorescent lesions in fundus images obtained by scanning laser ophthalmoscopy (AF-SLO) and by immunohistochemistry for Iba1 positive cells. The accumulation of subretinal macrophages was enhanced in Ccl2−/−, but not in Cx3cr1−/− or Ccl2−/−/Cx3cr1−/− mice. We identified no evidence of retinal degeneration in any of these mouse lines by TUNEL staining or semithin histology. In conclusion, CCL2-CCR2 and/or CX3CL1-CX3CR1 signalling defects may differentially affect the trafficking of microglia and macrophages in the retina during ageing, but do not appear to cause age-related retinal degeneration in mice.

Detection of Hard Exudates in Retinal Images Using a Radial Basis Function Classifier

Diabetic retinopathy (DR) is one of the most important causes of visual impairment. Automatic recognition of DR lesions, like hard exudates (EXs), in retinal images can contribute to the diagnosis and screening of the disease. The aim of this study was to automatically detect these lesions in fundus images. To achieve this goal, each image was normalized and the candidate EX regions were segmented by a combination of global and adaptive thresholding. Then, a group of features was extracted from image regions and the subset which best discriminated between EXs and retinal background was selected by means of logistic regression (LR). This optimal subset was subsequently used as input to a radial basis function (RBF) neural network. To improve the performance of the proposed algorithm, some noisy regions were eliminated by an innovative postprocessing of the image. The main novelty of the paper is the use of LR in conjunction with RBF and the proposed postprocessing technique. Our database was composed of 117 images with variable color, brightness and quality. The database was divided into a training set of 50 images (from DR patients) and a test set of 67 images (40 from DR patients and 27 from healthy retinas). Using a lesion-based criterion (pixel resolution), a mean sensitivity of 92.1% and a mean positive predictive value of 86.4% were obtained. With an image-based criterion, a mean sensitivity of 100%, mean specificity of 70.4% and mean accuracy of 88.1% were achieved. These results suggest that the proposed method could be a diagnostic aid for ophthalmologists in the screening for DR.

Neural network based detection of hard exudates in retinal images

Diabetic retinopathy (DR) is an important cause of visual impairment in developed countries. Automatic recognition of DR lesions in fundus images can contribute to the diagnosis of the disease. The aim of this study is to automatically detect one of these lesions, hard exudates (EXs), in order to help ophthalmologists in the diagnosis and follow-up of the disease. We propose an algorithm which includes a neural network (NN) classifier for this task. Three NN classifiers were investigated: multilayer perceptron (MLP), radial basis function (RBF) and support vector machine (SVM). Our database was composed of 117 images with variable colour, brightness, and quality. 50 of them (from DR patients) were used to train the NN classifiers and 67 (40 from DR patients and 27 from healthy retinas) to test the method. Using a lesion-based criterion, we achieved a mean sensitivity (SE l ) of 88.14% and a mean positive predictive value (PPV l ) of 80.72% for MLP. With RBF we obtained SE l = 88.49% and PPV l = 77.41%, while we reached SE l = 87.61% and PPV l = 83.51% using SVM. With an image-based criterion, a mean sensitivity (SE i ) of 100%, a mean specificity (SP i ) of 92.59% and a mean accuracy (AC i ) of 97.01% were obtained with MLP. Using RBF we achieved SE i = 100%, SP i = 81.48% and AC i = 92.54%. With SVM the image-based results were SE i = 100%, SP i = 77.78% and AC i = 91.04%.