Machine learning of retinal pathology in optical coherence tomography images

Central serous chorioretinopathy (CSC) is the fourth most common retinopathy and can reduce quality of life. CSC is assessed using optical coherence tomography (OCT), but deep learning systems have not been used to classify CSC subtypes. This study aimed to build a deep learning system model to distinguish CSC subtypes using a convolutional neural network (CNN). We enrolled 435 patients with CSC from a single tertiary center between January 2015 and January 2020. Data from spectral domain OCT (SD-OCT) images of the patients were analyzed using a deep CNN. Five-fold cross-validation was employed to evaluate the model’s ability to discriminate acute, non-resolving, inactive, and chronic atrophic CSC. We compared the performances of the proposed model, Resnet-50, Inception-V3, and eight ophthalmologists. Overall, 3209 SD-OCT images were included. The proposed model showed an average cross-validation accuracy of 70.0% (95% confidence interval [CI], 0.676–0.718) and the highest test accuracy was 73.5%. Additional evaluation in an independent set of 104 patients demonstrated the reliable performance of the proposed model (accuracy: 76.8%). Our model could classify CSC subtypes with high accuracy. Thus, automated deep learning systems could be useful in the classification and management of CSC.

Diagnosis retinal abnormalities in Optical Coherence Tomography (OCT) images assist ophthalmologist in the early detection and treatment of patients. To do this, different Computer Aided Diagnosis (CAD) methods based on machine learning and deep learning algorithms have been proposed. In this paper, wavelet scattering network is used to identify normal retina and four pathologies namely, Central Serous Retinopathy (CSR), Macular Hole (MH), Age-related Macular Degeneration (AMD) and Diabetic Retinopathy (DR). Wavelet scattering network is a particular convolutional network which is formed from cascading wavelet transform with nonlinear modulus and averaging operators. This transform generates sparse, translation invariant and deformation stable representations of signals. Filters in the layers of this network are predefined wavelets and not need to be learned which causes decreasing the processing time and complexity. The extracted features are fed to a Principal Component Analysis (PCA) classifier. The results of this research show the accuracy of 97.4% and 100% in diagnosis abnormal retina and DR from normal ones, respectively. We also achieved the accuracy of 84.2% in classifying OCT images to five classes of normal, CSR, MH, AMD and DR which outperforms other state of the art methods with high computational complexity. Clinical Relevance- Clinically, the manually checking of each OCT B-scan by ophthalmologists is tedious and time consuming and may lead to an erroneous decision specially for multiclass problems. In this study, a low complexity CAD system for retinal OCT image classification based on wavelet scattering network is introduced which can be learned by a small number of data.

The objective of this study was to build deep learning models with optical coherence tomography (OCT) images to classify normal and age related macular degeneration (AMD), AMD with fluid, and AMD without any fluid. In this study, 185 normal OCT images from 49 normal subjects, 535 OCT images of AMD with fluid, and 514 OCT mages of AMD without fluid from 120 AMD eyes as training data, while 49 normal images from 25 normal eyes, 188 AMD OCT images with fluid and 154 AMD images without any fluid from 77 AMD eyes as test data, were enrolled. Data augmentation was applied to increase the number of images to build deep learning models. Totally, two classification models were built in two steps. In the first step, a VGG16 model pre-trained on ImageNet dataset was transfer learned to classify normal and AMD, including AMD with fluid and/or without any fluid. Then, in the second step, the fine-tuned model in the first step was transfer learned again to distinguish the images of AMD with fluid from the ones without any fluid. With the first model, normal and AMD OCT images were classified with 0.999 area under receiver operating characteristic curve (AUC), and 99.2% accuracy. With the second model, AMD with the presence of any fluid, and AMD without fluid were classified with 0.992 AUC, and 95.1% accuracy. Compared with a transfer learned VGG16 model pre-trained on ImageNet dataset, to classify the three categories directly, higher classification performance was achieved with our notable approach. Conclusively, two classification models for AMD clinical practice were built with high classification performance, and these models should help improve the early diagnosis and treatment for AMD.

Alignment Artifacts in Optical Coherence Tomography Analyzed Images

Central Serous Retinopathy (CSR), also known as Central Serous Chorioretinopathy (CSC), occurs due to the clotting of fluids behind the retinal surface. The retina is composed of thin tissues that capture light and transform into visual recognition in the brain. This significant and critical organ may be damaged and causes vision loss and blindness for the individuals. Therefore, early-stage detection of the syndrome may cure complete loss of vision and, in some cases, may recover to its normal state. Hence, accurate and fast detection of CSR saves macula from severe damage and provides a basis for detecting other retinal pathologies. The Optical Coherence Tomographic (OCT) images have been used to detect CSR, but the design of a computationally efficient and accurate system remains a challenge. This research develops a framework for accurate and automatic CSR detection from OCT images using pre-trained deep convolutional neural networks. The preprocessing of OCT image enhances and filters the images for improving contrast and eliminate noise, respectively. Pre-trained network architectures have been employed, which are; AlexNet, ResNet-18, and GoogleNet for classification. The classification scheme followed by preprocessing enhances the foreground objects from OCT images. The performance of deep CNN has been compared through a statistical evaluation of parameters. The statistical parameters evaluation has shown 99.64% classification accuracy for AlexNet using Optical Coherence Tomography Image Database (OCTID). This shows the suitability of the proposed framework in clinical application to help doctors and clinicians diagnose retinal diseases.

This study examined and compared outcomes of deep learning (DL) in identifying swept-source optical coherence tomography (OCT) images without myopic macular lesions [i.e., no high myopia (nHM) vs. high myopia (HM)], and OCT images with myopic macular lesions [e.g., myopic choroidal neovascularization (mCNV) and retinoschisis (RS)]. A total of 910 SS-OCT images were included in the study as follows and analyzed by k-fold cross-validation (k = 5) using DL's renowned model, Visual Geometry Group-16: nHM, 146 images; HM, 531 images; mCNV, 122 images; and RS, 111 images (n = 910). The binary classification of OCT images with or without myopic macular lesions; the binary classification of HM images and images with myopic macular lesions (i.e., mCNV and RS images); and the ternary classification of HM, mCNV, and RS images were examined. Additionally, sensitivity, specificity, and the area under the curve (AUC) for the binary classifications as well as the correct answer rate for ternary classification were examined.The classification results of OCT images with or without myopic macular lesions were as follows: AUC, 0.970; sensitivity, 90.6%; specificity, 94.2%. The classification results of HM images and images with myopic macular lesions were as follows: AUC, 1.000; sensitivity, 100.0%; specificity, 100.0%. The correct answer rate in the ternary classification of HM images, mCNV images, and RS images were as follows: HM images, 96.5%; mCNV images, 77.9%; and RS, 67.6% with mean, 88.9%.Using noninvasive, easy-to-obtain swept-source OCT images, the DL model was able to classify OCT images without myopic macular lesions and OCT images with myopic macular lesions such as mCNV and RS with high accuracy. The study results suggest the possibility of conducting highly accurate screening of ocular diseases using artificial intelligence, which may improve the prevention of blindness and reduce workloads for ophthalmologists.

This study examined and compared outcomes of deep learning (DL) in identifying swept-source optical coherence tomography (OCT) images without myopic macular lesions [i.e., no high myopia (nHM) vs. high myopia (HM)], and OCT images with myopic macular lesions [e.g., myopic choroidal neovascularization (mCNV) and retinoschisis (RS)]. A total of 910 SS-OCT images were included in the study as follows and analyzed by k-fold cross-validation (k = 5) using DL's renowned model, Visual Geometry Group-16: nHM, 146 images; HM, 531 images; mCNV, 122 images; and RS, 111 images (n = 910). The binary classification of OCT images with or without myopic macular lesions; the binary classification of HM images and images with myopic macular lesions (i.e., mCNV and RS images); and the ternary classification of HM, mCNV, and RS images were examined. Additionally, sensitivity, specificity, and the area under the curve (AUC) for the binary classifications as well as the correct answer rate for ternary classification were examined. The classification results of OCT images with or without myopic macular lesions were as follows: AUC, 0.970; sensitivity, 90.6%; specificity, 94.2%. The classification results of HM images and images with myopic macular lesions were as follows: AUC, 1.000; sensitivity, 100.0%; specificity, 100.0%. The correct answer rate in the ternary classification of HM images, mCNV images, and RS images were as follows: HM images, 96.5%; mCNV images, 77.9%; and RS, 67.6% with mean, 88.9%.Using noninvasive, easy-to-obtain swept-source OCT images, the DL model was able to classify OCT images without myopic macular lesions and OCT images with myopic macular lesions such as mCNV and RS with high accuracy. The study results suggest the possibility of conducting highly accurate screening of ocular diseases using artificial intelligence, which may improve the prevention of blindness and reduce workloads for ophthalmologists.

Fracture detection by artificial intelligence and especially Deep Convolutional Neural Networks (DCNN) is a topic of growing interest in current orthopaedic and radiological research. As learning a DCNN usually needs a large amount of training data, mostly frequent fractures as well as conventional X-ray are used. Therefore, less common fractures like acetabular fractures (AF) are underrepresented in the literature. The aim of this pilot study was to establish a DCNN for detection of AF using computer tomography (CT) scans. Patients with an acetabular fracture were identified from the monocentric consecutive pelvic injury registry at the BG Trauma Center XXX from 01/2003 - 12/2019. All patients with unilateral AF and CT scans available in DICOM-format were included for further processing. All datasets were automatically anonymised and digitally post-processed. Extraction of the relevant region of interests was performed and the technique of data augmentation (DA) was implemented to artificially increase the number of training samples. A DCNN based on Med3D was used for autonomous fracture detection, using global average pooling (GAP) to reduce overfitting. From a total of 2,340 patients with a pelvic fracture, 654 patients suffered from an AF. After screening and post-processing of the datasets, a total of 159 datasets were enrolled for training of the algorithm. A random assignment into training datasets (80%) and test datasets (20%) was performed. The technique of bone area extraction, DA and GAP increased the accuracy of fracture detection from 58.8% (native DCNN) up to an accuracy of 82.8% despite the low number of datasets. The accuracy of fracture detection of our trained DCNN is comparable to published values despite the low number of training datasets. The techniques of bone extraction, DA and GAP are useful for increasing the detection rates of rare fractures by a DCNN. Based on the used DCNN in combination with the described techniques from this pilot study, the possibility of an automatic fracture classification of AF is under investigation in a multicentre study.

Background Fracture detection by artificial intelligence and especially Deep Convolutional Neural Networks (DCNN) is a topic of growing interest in current orthopaedic and radiological research. As learning a DCNN usually needs a large amount of training data, mostly frequent fractures as well as conventional X-ray are used. Therefore, less common fractures like acetabular fractures (AF) are underrepresented in the literature. The aim of this pilot study was to establish a DCNN for detection of AF using computer tomography (CT) scans. Methods Patients with an acetabular fracture were identified from the monocentric consecutive pelvic injury registry at the BG Trauma Center XXX from 01/2003–12/2019. All patients with unilateral AF and CT scans available in DICOM-format were included for further processing. All datasets were automatically anonymised and digitally post-processed. Extraction of the relevant region of interests was performed and the technique of data augmentation (DA) was implemented to artificially increase the number of training samples. A DCNN based on Med3D was used for autonomous fracture detection, using global average pooling (GAP) to reduce overfitting. Results From a total of 2,340 patients with a pelvic fracture, 654 patients suffered from an AF. After screening and post-processing of the datasets, a total of 159 datasets were enrolled for training of the algorithm. A random assignment into training datasets (80%) and test datasets (20%) was performed. The technique of bone area extraction, DA and GAP increased the accuracy of fracture detection from 58.8% (native DCNN) up to an accuracy of 82.8% despite the low number of datasets. Conclusion The accuracy of fracture detection of our trained DCNN is comparable to published values despite the low number of training datasets. The techniques of bone extraction, DA and GAP are useful for increasing the detection rates of rare fractures by a DCNN. Based on the used DCNN in combination with the described techniques from this pilot study, the possibility of an automatic fracture classification of AF is under investigation in a multicentre study.

Driver drowsiness is one of the main causes of driver-related motor vehicle collisions, as this impairs a person’s concentration whilst driving. With the enhancements of computer vision and deep learning (DL), driver drowsiness detection systems have been developed previously, in an attempt to improve road safety. These systems experienced performance degradation under real-world testing due to factors such as driver movement and poor lighting. This study proposed to improve the training of DL models for driver drowsiness detection by applying data augmentation (DA) techniques that model these real-world scenarios. This paper studies six DL models for driver drowsiness detection: four configurations of a Convolutional Neural Network (CNN), two custom configurations as well as the architectures designed by the Visual Geometry Group (VGG) (i.e. VGG16 and VGG19); a Generative Adversarial Network (GAN) and a Multi-Layer Perceptron (MLP). These DL models were trained using two datasets of eye images, where the state of eye (open or closed) is used in determining driver drowsiness. The performance of the DL models was measured with respect to accuracy, F1-Score, precision, negative class precision, recall and specificity. When comparing the performance of DL models trained on datasets with and without DA in aggregation, it was found that all metrics were improved. After removing outliers from the results, it was found that the average improvement in both accuracy and F1 score due to DA was +4.3%. Furthermore, it is shown that the extent to which the DA techniques improve DL model performance is correlated with the inherent model performance. For DL models with accuracy and F1-Score ≤ 90%, results show that the DA techniques studied should improve performance by at least +5%.

Neovascular age-related macular degeneration (nAMD) is among the main causes of visual impairment worldwide. We built a deep learning model to distinguish the subtypes of nAMD using spectral domain optical coherence tomography (SD-OCT) images. Data from SD-OCT images of nAMD (polypoidal choroidal vasculopathy, retinal angiomatous proliferation, and typical nAMD) and normal healthy patients were analyzed using a convolutional neural network (CNN). The model was trained and validated based on 4749 SD-OCT images from 347 patients and 50 healthy controls. To adopt an accurate and robust image classification architecture, we evaluated three well-known CNN structures (VGG-16, VGG-19, and ResNet) and two customized classification layers (fully connected layer with dropout vs. global average pooling). Following the test set performance, the model with the highest classification accuracy was used. Transfer learning and data augmentation were applied to improve the robustness and accuracy of the model. Our proposed model showed an accuracy of 87.4% on the test data (920 images), scoring higher than ten ophthalmologists, for the same data. Additionally, the part that our model judged to be important in classification was confirmed through Grad-CAM images, and consequently, it has a similar judgment criteria to that of ophthalmologists. Thus, we believe that our model can be used as an auxiliary tool in clinical practice.

Optical coherence tomography (OCT) imaging is a well-known technology for visualizing retinal layers and helps ophthalmologists to detect possible diseases. Accurate and early diagnosis of common retinal diseases can prevent the patients from suffering critical damages to their vision. Computer-aided diagnosis (CAD) systems can significantly assist ophthalmologists in improving their examinations. This paper presents a new enhanced deep ensemble convolutional neural network for detecting retinal diseases from OCT images. Our model generates rich and multi-resolution features by employing the learning architectures of two robust convolutional models. Spatial resolution is a critical factor in medical images, especially the OCT images that contain tiny essential points. To empower our model, we apply a new post-architecture model to our ensemble model for enhancing spatial resolution learning without increasing computational costs. The introduced post-architecture model can be deployed to any feature extraction model to improve the utilization of the feature map’s spatial values. We have collected two open-source datasets for our experiments to make our models capable of detecting six crucial retinal diseases: Age-related Macular Degeneration (AMD), Central Serous Retinopathy (CSR), Diabetic Retinopathy (DR), Choroidal Neovascularization (CNV), Diabetic Macular Edema (DME), and Drusen alongside the normal cases. Our experiments on two datasets and comparing our model with some other well-known deep convolutional neural networks have proven that our architecture can increase the classification accuracy up to 5%. We hope that our proposed methods create the next step of CAD systems development and help future researches.

We present an automatic classification algorithm for retinal optical coherence tomography (OCT) images based on convolution neural network (CNN). This algorithm inherently contains feature extraction and classification, thus avoiding the design feature extractor manually. Firstly, we processed the OCT images to focus on and determine the pathological area of the retinal OCT images, and to speed up the training of the network. Then we input the original images to crop them, which can effectively prevent the noise introduced in the processes of image processing and changing the pixels in the original image. Secondly, we augmented the OCT images in the source data set to obtain sufficient images, and to alleviate the impact of a relatively small number of target classification images on the model accuracy and generalization ability. Our method was introduced the random translation in image cropping and horizontal flipped to augment the OCT images. Then we applied two methods to build two data sets used to train the network, and we divided each of the data sets into a training set and a validation set. Next, we designed an efficient classification network and trained it with the two training sets respectively, to acquire the two models that can classify OCT images. The results indicate that the network trained by the augmented data can classify images more effectively. In our classification algorithm, the accuracy, the sensitivity and the specificity are 93.43%, 91.38%, and 95.88%, respectively.

IntroductionThe use of optical coherence tomography (OCT) images is increasing in the medical treatment of age-related macular degeneration (AMD), and thus, the amount of data requiring analysis is increasing. Advances in machine-learning techniques may facilitate processing of large amounts of medical image data. Among deep-learning methods, convolution neural networks (CNNs) show superior image recognition ability. This study aimed to build deep-learning models that could distinguish AMD from healthy OCT scans and to distinguish AMD with and without exudative changes without using a segmentation algorithm.MethodsThis was a cross-sectional observational clinical study. A total of 1621 spectral domain (SD)-OCT images of patients with AMD and a healthy control group were studied. The first CNN model was trained and validated using 1382 AMD images and 239 normal images. The second transfer-learning model was trained and validated with 721 AMD images with exudative changes and 661 AMD images without any exudate. The attention area of the CNN was described as a heat map by class activation mapping (CAM). In the second model, which classified images into AMD with or without exudative changes, we compared the learning stabilization of models using or not using transfer learning.ResultsUsing the first CNN model, we could classify AMD and normal OCT images with 100% sensitivity, 91.8% specificity, and 99.0% accuracy. In the second, transfer-learning model, we could classify AMD as having or not having exudative changes, with 98.4% sensitivity, 88.3% specificity, and 93.9% accuracy. CAM successfully described the heat-map area on the OCT images. Including the transfer-learning model in the second model resulted in faster stabilization than when the transfer-learning model was not included.ConclusionTwo computational deep-learning models were developed and evaluated here; both models showed good performance. Automation of the interpretation process by using deep-learning models can save time and improve efficiency.Trial RegistrationNo15073.

Maculopathy is the excessive damage to macula that leads to blindness. It mostly occurs due to retinal edema (RE), central serous chorioretinopathy (CSCR), or age related macular degeneration (ARMD). Optical coherence tomography (OCT) imaging is the latest eye testing technique that can detect these syndromes in early stages. Many researchers have used OCT images to detect retinal abnormalities. However, to the best of our knowledge, no research that presents a fully automated system to detect all of these macular syndromes is reported. This paper presents the world's first ever decision support system to automatically detect RE, CSCR, and ARMD retinal pathologies and healthy retina from OCT images. The automated disease diagnosis in our proposed system is based on multilayered support vector machines (SVM) classifier trained on 40 labeled OCT scans (10 healthy, 10 RE, 10 CSCR, and 10 ARMD). After training, SVM forms an accurate decision about the type of retinal pathology using 9 extracted features. We have tested our proposed system on 2819 OCT scans (1437 healthy, 640 RE, and 742 CSCR) of 502 patients from two different datasets and our proposed system correctly diagnosed 2817/2819 subjects with the accuracy, sensitivity, and specificity ratings of 99.92%, 100%, and 99.86%, respectively.