Skin strata delineation in reflectance confocal microscopy images using recurrent convolutional networks with attention

Alican Bozkurt,Dana H Brooks,Jennifer G Dy,Milind Rajadhyaksha,Kivanc Kose,Jaume Coll-Font,Christi Alessi-Fox

doi:10.1038/s41598-021-90328-x

Abstract

Reflectance confocal microscopy (RCM) is an effective non-invasive tool for cancer diagnosis. However, acquiring and reading RCM images requires extensive training and experience, and novice clinicians exhibit high discordance in diagnostic accuracy. Quantitative tools to standardize image acquisition could reduce both required training and diagnostic variability. To perform diagnostic analysis, clinicians collect a set of RCM mosaics (RCM images concatenated in a raster fashion to extend the field view) at 4–5 specific layers in skin, all localized in the junction between the epidermal and dermal layers (dermal-epidermal junction, DEJ), necessitating locating that junction before mosaic acquisition. In this study, we automate DEJ localization using deep recurrent convolutional neural networks to delineate skin strata in stacks of RCM images collected at consecutive depths. Success will guide to automated and quantitative mosaic acquisition thus reducing inter operator variability and bring standardization in imaging. Testing our model against an expert labeled dataset of 504 RCM stacks, we achieved 88.07% classification accuracy and nine-fold reduction in the number of anatomically impossible errors compared to the previous state-of-the-art.

Highlights

Reflectance confocal microscopy (RCM) is an effective non-invasive tool for cancer diagnosis
Our solution, which we report in this paper, is to first train a deep convolutional neural network (CNN) to classify individual RCM images as epidermis, dermal-epidermal junction (DEJ), or dermis and to exploit the sequential structure of skin layers by augmenting the CNN with recurrent neural network (RNN) layers
The others all involved different deep neural networks (DNNs). These networks differed in the basic DNN structure, in whether they included an attention mechanism[31,32] or not, in whether they included recurrent neural network (RNN) components (e.g. gated recurrent units[33] (GRUs)) or not, and in whether they used only the single slice of interest, a local neighbourhood of the slice, or the entire stack

Summary

Introduction

Reflectance confocal microscopy (RCM) is an effective non-invasive tool for cancer diagnosis. Macroscopic imaging techniques in dermatology (e.g. dermoscopy, clinical photography) are typically surface imaging techniques that are capable of collecting color images of skin lesions They can image entire volume of skin lesions without the ability to visualize individual layers of cells (i.e, without depth-resolution). Higher resolution in vivo microscopic imaging techniques are capable of collecting thin en-face optical sections with depth-resolution, allowing for imaging of individual cell layers within a 3D volume (from skin surface to the dermis), with cellular resolution that is not provided by macroscopic imaging techniques Among those higher resolution in vivo microscopy technologies, reflectance confocal microscopy (RCM) has begun to play a unique role in diagnostic dermatology. This has been proven valuable for lesions which lack distinct visual features and patterns and cannot be diagnosed with dermoscopy

Methods

Results

Discussion

Conclusion