Abstract
A resolution enhancement technique for optical coherence tomography (OCT), based on Generative Adversarial Networks (GANs), was developed and investigated. GANs have been previously used for resolution enhancement of photography and optical microscopy images. We have adapted and improved this technique for OCT image generation. Conditional GANs (cGANs) were trained on a novel set of ultrahigh resolution spectral domain OCT volumes, termed micro-OCT, as the high-resolution ground truth (∼1 μm isotropic resolution). The ground truth was paired with a low-resolution image obtained by synthetically degrading resolution 4x in one of (1-D) or both axial and lateral axes (2-D). Cross-sectional image (B-scan) volumes obtained from in vivo imaging of human labial (lip) tissue and mouse skin were used in separate feasibility experiments. Accuracy of resolution enhancement compared to ground truth was quantified with human perceptual accuracy tests performed by an OCT expert. The GAN loss in the optimization objective, noise injection in both the generator and discriminator models, and multi-scale discrimination were found to be important for achieving realistic speckle appearance in the generated OCT images. The utility of high-resolution speckle recovery was illustrated by an example of micro-OCT imaging of blood vessels in lip tissue. Qualitative examples applying the models to image data from outside of the training data distribution, namely human retina and mouse bladder, were also demonstrated, suggesting potential for cross-domain transferability. This preliminary study suggests that deep learning generative models trained on OCT images from high-performance prototype systems may have potential in enhancing lower resolution data from mainstream/commercial systems, thereby bringing cutting-edge technology to the masses at low cost.
Highlights
Optical coherence tomography (OCT) is a 3-dimensional optical imaging technique, and has become part of the standard of care in ophthalmology [1] while growing in importance in other clinical specialties such as gastroenterology [2]
In this work we explore the hypothesis that Conditional GANs (cGANs) can be used to enhance the optical axial and lateral resolution of OCT images while preserving and improving the detail of speckle content, trained on an ultrahigh resolution OCT ground truth
We have developed a deep learning based algorithm for resolution enhancement of OCT images, based on previously reported techniques in generative adversarial networks
Summary
Optical coherence tomography (OCT) is a 3-dimensional optical imaging technique, and has become part of the standard of care in ophthalmology [1] while growing in importance in other clinical specialties such as gastroenterology [2]. We identified critically important modifications to a conventional conditional GAN framework for producing high quality resolution enhancement and realistic speckle generation in OCT images, namely noise injection and multi-scale discrimination, 2. As training of the generator and discriminator models is performed alternately, the two models compete till a theoretical limit where the generated images are indistinguishable from the ground truth, in practice the generated quality does not necessarily converge to an optimum This previously reported cGAN design is widely known as ‘pix2pix’ [20] and has several open-source skeleton implementations generously made available by the machine learning community [27, 28]. Even though model training was performed on image patches, the fully convolutional nature of the generator model (with no fully connected layers) allowed the use of image inputs that were larger than and not restricted to the training patch size of 256x256, the full-sized original images could be used in the generator at model prediction time
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
