Skin Biopsy Images Research Articles

Sarcoidosis and ulcerative colitis: overlap or coexistence.

We show an unusual case of ulcerative colitis and sarcoidosis. This case has detailed images of skin lesions, biopsies and PET-CT results. There is limited evidence regarding the association of inflammatory bowel disease and other granulomatous diseases, especially sarcoidosis.

Automated analysis of whole slide digital skin biopsy images.

A rapidly increasing rate of melanoma diagnosis has been noted over the past three decades, and nearly 1 in 4 skin biopsies are diagnosed as melanocytic lesions. The gold standard for diagnosis of melanoma is the histopathological examination by a pathologist to analyze biopsy material at both the cellular and structural levels. A pathologist's diagnosis is often subjective and prone to variability, while deep learning image analysis methods may improve and complement current diagnostic and prognostic capabilities. Mitoses are important entities when reviewing skin biopsy cases as their presence carries prognostic information; thus, their precise detection is an important factor for clinical care. In addition, semantic segmentation of clinically important structures in skin biopsies might help the diagnosis pipeline with an accurate classification. We aim to provide prognostic and diagnostic information on skin biopsy images, including the detection of cellular level entities, segmentation of clinically important tissue structures, and other important factors toward the accurate diagnosis of skin biopsy images. This paper is an overview of our work on analysis of digital whole slide skin biopsy images, including mitotic figure (mitosis) detection, semantic segmentation, diagnosis, and analysis of pathologists' viewing patterns, and with new work on melanocyte detection. Deep learning has been applied to our methods for all the detection, segmentation, and diagnosis work. In our studies, deep learning is proven superior to prior approaches to skin biopsy analysis. Our work on analysis of pathologists' viewing patterns is the only such work in the skin biopsy literature. Our work covers the whole spectrum from low-level entities through diagnosis and understanding what pathologists do in performing their diagnoses.

Improving the Diagnosis of Skin Biopsies Using Tissue Segmentation.

Invasive melanoma, a common type of skin cancer, is considered one of the deadliest. Pathologists routinely evaluate melanocytic lesions to determine the amount of atypia, and if the lesion represents an invasive melanoma, its stage. However, due to the complicated nature of these assessments, inter- and intra-observer variability among pathologists in their interpretation are very common. Machine-learning techniques have shown impressive and robust performance on various tasks including healthcare. In this work, we study the potential of including semantic segmentation of clinically important tissue structure in improving the diagnosis of skin biopsy images. Our experimental results show a 6% improvement in F-score when using whole slide images along with epidermal nests and cancerous dermal nest segmentation masks compared to using whole-slide images alone in training and testing the diagnosis pipeline.

An analysis of pathologists’ viewing processes as they diagnose whole slide digital images

Although pathologists have their own viewing habits while diagnosing, viewing behaviors leading to the most accurate diagnoses are under-investigated. Digital whole slide imaging has enabled investigators to analyze pathologists’ visual interpretation of histopathological features using mouse and viewport tracking techniques. In this study, we provide definitions for basic viewing behavior variables and investigate the association of pathologists' characteristics and viewing behaviors, and how they relate to diagnostic accuracy when interpreting whole slide images. We use recordings of 32 pathologists’ actions while interpreting a set of 36 digital whole slide skin biopsy images (5 sets of 36 cases; 180 cases total). These viewport tracking data include the coordinates of a viewport scene on pathologists’ screens, the magnification level at which that viewport was viewed, as well as a timestamp. We define a set of variables to quantify pathologists' viewing behaviors such as zooming, panning, and interacting with a consensus reference panel’s selected region of interest (ROI). We examine the association of these viewing behaviors with pathologists’ demographics, clinical characteristics, and diagnostic accuracy using cross-classified multilevel models. Viewing behaviors differ based on clinical experience of the pathologists. Pathologists with a higher caseload of melanocytic skin biopsy cases and pathologists with board certification and/or fellowship training in dermatopathology have lower average zoom and lower variance of zoom levels. Viewing behaviors associated with higher diagnostic accuracy include higher average and variance of zoom levels, a lower magnification percentage (a measure of consecutive zooming behavior), higher total interpretation time, and higher amount of time spent viewing ROIs. Scanning behavior, which refers to panning with a fixed zoom level, has marginally significant positive association with accuracy. Pathologists’ training, clinical experience, and their exposure to a range of cases are associated with their viewing behaviors, which may contribute to their diagnostic accuracy. Research in computational pathology integrating digital imaging and clinical informatics opens up new avenues for leveraging viewing behaviors in medical education and training, potentially improving patient care and the effectiveness of clinical workflow.

MICaps: Multi-instance capsule network for machine inspection of Munro's microabscess

Munro's Microabscess (MM) is the diagnostic hallmark of psoriasis. Neutrophil detection in the Stratum Corneum (SC) of the skin epidermis is an integral part of MM detection in skin biopsy. The microscopic inspection of skin biopsy is a tedious task and staining variations in skin histopathology often hinder human performance to differentiate neutrophils from skin keratinocytes. Motivated from this, we propose a computational framework that can assist human experts and reduce potential errors in diagnosis. The framework first segments the SC layer, and multiple patches are sampled from the segmented regions which are classified to detect neutrophils. Both UNet and CapsNet are used for segmentation and classification. Experiments show that of the two choices, CapsNet, owing to its robustness towards better hierarchical object representation and localisation ability, appears as a better candidate for both segmentation and classification tasks and hence, we termed our framework as MICaps. The training algorithm explores both minimisation of Dice Loss and Focal Loss and makes a comparative study between the two. The proposed framework is validated with our in-house dataset consisting of 290 skin biopsy images. Two different experiments are considered. Under the first protocol, only 3-fold cross-validation is done to directly compare the current results with the state-of-the-art ones. Next, the performance of the system on a held-out data set is reported. The experimental results show that MICaps improves the state-of-the-art diagnosis performance by 3.27% (maximum) and reduces the number of model parameters by 50%.

Melanoma in the blink of an eye: Pathologists’ rapid detection, classification, and localization of skin abnormalities

ABSTRACT Expert radiologists can quickly extract a basic “gist” understanding of a medical image following less than a second exposure, leading to above-chance diagnostic classification of images. It is currently unclear whether this pattern of results is applicable to pathology, another medical imaging domain demanding visual diagnostic interpretation. To further characterize the rapid interpretation medical images, this study examined eye movements and diagnostic decision-making when pathologists were briefly exposed to digital whole slide images of melanocytic skin biopsies. Twelve resident (N = 5), fellow (N = 5), and attending pathologists (N = 2) with experience interpreting dermatopathology briefly viewed 48 cases presented for 500 ms each, and we tracked their eye movements towards abnormalities, their ability to classify images as containing or not containing invasive melanoma, and their ability to localize critical image regions. Results demonstrated rapid shifts of the eyes towards critical abnormalities, high diagnostic sensitivity and specificity, and surprisingly high accuracy localizing critical diagnostic regions. Furthermore, when pathologists fixated critical regions with their eyes, they were subsequently more likely to successfully localize that region on an outline of the image. Results are discussed relative to models of medical image interpretation and innovative methods for monitoring and assessing expertise development during medical education and training.

Scale-Aware Transformers for Diagnosing Melanocytic Lesions

Diagnosing melanocytic lesions is one of the most challenging areas of pathology with extensive intra- and inter-observer variability. The gold standard for a diagnosis of invasive melanoma is the examination of histopathological whole slide skin biopsy images by an experienced dermatopathologist. Digitized whole slide images offer novel opportunities for computer programs to improve the diagnostic performance of pathologists. In order to automatically classify such images, representations that reflect the content and context of the input images are needed. In this paper, we introduce a novel self-attention-based network to learn representations from digital whole slide images of melanocytic skin lesions at multiple scales. Our model softly weighs representations from multiple scales, allowing it to discriminate between diagnosis-relevant and -irrelevant information automatically. Our experiments show that our method outperforms five other state-of-the-art whole slide image classification methods by a significant margin. Our method also achieves comparable performance to 187 practicing U.S. pathologists who interpreted the same cases in an independent study. To facilitate relevant research, full training and inference code is made publicly available at https://github.com/meredith-wenjunwu/ScATNet.

Machine learning techniques for mitoses classification

BackgroundPathologists analyze biopsy material at both the cellular and structural level to determine diagnosis and cancer stage. Mitotic figures are surrogate biomarkers of cellular proliferation that can provide prognostic information; thus, their precise detection is an important factor for clinical care. Convolutional Neural Networks (CNNs) have shown remarkable performance on several recognition tasks. Utilizing CNNs for mitosis classification may aid pathologists to improve the detection accuracy. MethodsWe studied two state-of-the-art CNN-based models, ESPNet and DenseNet, for mitosis classification on six whole slide images of skin biopsies and compared their quantitative performance in terms of sensitivity, specificity, and F-score. We used raw RGB images of mitosis and non-mitosis samples with their corresponding labels as training input. In order to compare with other work, we studied the performance of these classifiers and two other architectures, ResNet and ShuffleNet, on the publicly available MITOS breast biopsy dataset and compared the performance of all four in terms of precision, recall, and F-score (which are standard for this data set), architecture, training time and inference time. ResultsThe ESPNet and DenseNet results on our primary melanoma dataset had a sensitivity of 0.976 and 0.968, and a specificity of 0.987 and 0.995, respectively, with F-scores of .968 and .976, respectively. On the MITOS dataset, ESPNet and DenseNet showed a sensitivity of 0.866 and 0.916, and a specificity of 0.973 and 0.980, respectively. The MITOS results using DenseNet had a precision of 0.939, recall of 0.916, and F-score of 0.927. The best published result on MITOS (Saha et al. 2018) reported precision of 0.92, recall of 0.88, and F-score of 0.90. In our architecture comparisons on MITOS, we found that DenseNet beats the others in terms of F-Score (DenseNet 0.927, ESPNet 0.890, ResNet 0.865, ShuffleNet 0.847) and especially Recall (DenseNet 0.916, ESPNet 0.866, ResNet 0.807, ShuffleNet 0.753), while ResNet and ESPNet have much faster inference times (ResNet 6 s, ESPNet 8 s, DenseNet 31 s). ResNet is faster than ESPNet, but ESPNet has a higher F-Score and Recall than ResNet, making it a good compromise solution. ConclusionWe studied several state-of-the-art CNNs for detecting mitotic figures in whole slide biopsy images. We evaluated two CNNs on a melanoma cancer dataset and then compared four CNNs on a public breast cancer data set, using the same methodology on both. Our methodology and architecture for mitosis finding in both melanoma and breast cancer whole slide images has been thoroughly tested and is likely to be useful for finding mitoses in any whole slide biopsy images.

Nonscanning large-area Raman imaging for ex vivo/in vivo skin cancer discrimination.

Imaging Raman spectroscopy can be used to identify cancerous tissue. Traditionally, a step-by-step scanning of the sample is applied to generate a Raman image, which, however, is too slow for routine examination of patients. By transferring the technique of integral field spectroscopy (IFS) from astronomy to Raman imaging, it becomes possible to record entire Raman images quickly within a single exposure, without the need for a tedious scanning procedure. An IFS-based Raman imaging setup is presented, which is capable of measuring skin ex vivo or in vivo. It is demonstrated how Raman images of healthy and cancerous skin biopsies were recorded and analyzed.

Dermal fiber structures and photoaging.

The use of multiphoton imaging has become a standard technique to visualize the dermis fibers as it requires no specific staining. The density and organization of collagen and elastin are common markers of skin intrinsic aging and photoaging; thus, there is a need of grading this skin aging with quantitative indicators able to provide a robust evaluation of the dermis fibers' state. We propose a systematic analysis of multiphoton images of skin biopsies taken on the buttock and the forearm of patients of different ages. The intensity histograms of images were analyzed through their moments, a wavelet decomposition was done, and the wavelet coefficients distribution was fitted by a generalized Gaussian distribution. Different parameters relative to the collagen or elastin densities, organizations, and structures were calculated and exhibit phenomena specific to intrinsic or extrinsic aging. Those indicators could become a standard method to analyze the degree of skin aging (intrinsic or extrinsic) through multiphoton imaging.

Automated analysis and classification of melanocytic tumor on skin whole slide images

This paper presents a computer-aided technique for automated analysis and classification of melanocytic tumor on skin whole slide biopsy images. The proposed technique consists of four main modules. First, skin epidermis and dermis regions are segmented by a multi-resolution framework. Next, epidermis analysis is performed, where a set of epidermis features reflecting nuclear morphologies and spatial distributions is computed. In parallel with epidermis analysis, dermis analysis is also performed, where dermal cell nuclei are segmented and a set of textural and cytological features are computed. Finally, the skin melanocytic image is classified into different categories such as melanoma, nevus or normal tissue by using a multi-class support vector machine (mSVM) with extracted epidermis and dermis features. Experimental results on 66 skin whole slide images indicate that the proposed technique achieves more than 95% classification accuracy, which suggests that the technique has the potential to be used for assisting pathologists on skin biopsy image analysis and classification.

Psoriasis skin biopsy image segmentation using Deep Convolutional Neural Network

Background and objectiveDevelopment of machine assisted tools for automatic analysis of psoriasis skin biopsy image plays an important role in clinical assistance. Development of automatic approach for accurate segmentation of psoriasis skin biopsy image is the initial prerequisite for developing such system. However, the complex cellular structure, presence of imaging artifacts, uneven staining variation make the task challenging. This paper presents a pioneering attempt for automatic segmentation of psoriasis skin biopsy images. MethodsSeveral deep neural architectures are tried for segmenting psoriasis skin biopsy images. Deep models are used for classifying the super-pixels generated by Simple Linear Iterative Clustering (SLIC) and the segmentation performance of these architectures is compared with the traditional hand-crafted feature based classifiers built on popularly used classifiers like K-Nearest Neighbor (KNN), Support Vector Machine (SVM) and Random Forest (RF). A U-shaped Fully Convolutional Neural Network (FCN) is also used in an end to end learning fashion where input is the original color image and the output is the segmentation class map for the skin layers. ResultsAn annotated real psoriasis skin biopsy image data set of ninety (90) images is developed and used for this research. The segmentation performance is evaluated with two metrics namely, Jaccard’s Coefficient (JC) and the Ratio of Correct Pixel Classification (RCPC) accuracy. The experimental results show that the CNN based approaches outperform the traditional hand-crafted feature based classification approaches. ConclusionsThe present research shows that practical system can be developed for machine assisted analysis of psoriasis disease.

3D imaging of cleared human skin biopsies using light-sheet microscopy: A new way to visualize in-depth skin structure.

Human skin is composed of the superimposition of tissue layers of various thicknesses and components. Histological staining of skin sections is the benchmark approach to analyse the organization and integrity of human skin biopsies; however, this approach does not allow 3D tissue visualization. Alternatively, confocal or two-photon microscopy is an effective approach to perform fluorescent-based 3D imaging. However, owing to light scattering, these methods display limited light penetration in depth. The objectives of this study were therefore to combine optical clearing and light-sheet fluorescence microscopy (LSFM) to perform in-depth optical sectioning of 5mm-thick human skin biopsies and generate 3D images of entire human skin biopsies. A benzyl alcohol and benzyl benzoate solution was used to successfully optically clear entire formalin fixed human skin biopsies, making them transparent. In-depth optical sectioning was performed with LSFM on the basis of tissue-autofluorescence observations. 3D image analysis of optical sections generated with LSFM was performed by using the Amira® software. This new approach allowed us to observe in situ the different layers and compartments of human skin, such as the stratum corneum, the dermis and epidermal appendages. With this approach, we easily performed 3D reconstruction to visualise an entire human skin biopsy. Finally, we demonstrated that this method is useful to visualise and quantify histological anomalies, such as epidermal hyperplasia. The combination of optical clearing and LSFM has new applications in dermatology and dermatological research by allowing 3D visualization and analysis of whole human skin biopsies.

Deep learning based feature representation for automated skin histopathological image annotation

Automated annotation of skin biopsy histopathological images provides valuable information and supports for diagnosis, especially for the discrimination between malignant and benign lesions. Currently, computer-aid analysis of skin biopsy images mostly relied on some human-designed features, which requires expensive human efforts and experiences in problem domains. In this study, we propose an annotation framework for automated skin biopsy image analysis which makes use of a deep model for image feature representation. A convolutional neural network (CNN) is designed for local regions of skin biopsy images which learns potential high-level features automatically from input raw pixels. The annotation model is constructed in the multiple-instance multiple-label (MIML) learning framework with the features learned through the network. We achieve significant improvement of the model performance on a real world clinical skin biopsy image dataset and a benchmark dataset. Moreover, our study indicates that deep learning based model could achieve better performance than human designed features.

Dermal type I collagen assessment by digital image analysis.

Type I collagen is the main dermal component, and its evaluation is relevant to quantitative studies in dermatopathology. However, visual gradation (0 to 4+) has low precision and high subjectivity levels. This study aimed to develop and validate a digital morphometric analysis technique to estimate type I collagen levels in the papillary dermis. Four evaluators visually quantified (0 to 4+) the density of type I collagen in 63 images of forearm skin biopsies marked by immunohistochemistry and two evaluators analyzed the same images using digital morphometric techniques (RGB split colors (I) and color deconvolution (II)). Automated type I collagen density estimation in the papillary dermis (two techniques) were correlated with visual evaluations (Spearman's rho coefficients of 0.48 and 0.62 (p<0.01)). With regard to the inter-observer repeatability, the four evaluators who used visual classification had an intraclass correlation coefficient (for absolute agreement) of 0.53, while the other two evaluators who used digital analysis (algorithm II) had an intraclass correlation coefficient of 0.97.

Augmenting multi-instance multilabel learning with sparse bayesian models for skin biopsy image analysis.

Skin biopsy images can reveal causes and severity of many skin diseases, which is a significant complement for skin surface inspection. Automatic annotation of skin biopsy image is an important problem for increasing efficiency and reducing the subjectiveness in diagnosis. However it is challenging particularly when there exists indirect relationship between annotation terms and local regions of a biopsy image, as well as local structures with different textures. In this paper, a novel method based on a recent proposed machine learning model, named multi-instance multilabel (MIML), is proposed to model the potential knowledge and experience of doctors on skin biopsy image annotation. We first show that the problem of skin biopsy image annotation can naturally be expressed as a MIML problem and then propose an image representation method that can capture both region structure and texture features, and a sparse Bayesian MIML algorithm which can produce probabilities indicating the confidence of annotation. The proposed algorithm framework is evaluated on a real clinical dataset containing 12,700 skin biopsy images. The results show that it is effective and prominent.

A ‘migrant’ mass of the forehead: Diagnosis and treatment

Primary cutaneous large B-cell lymphoma, leg type, is a rare and aggressive neoplasm as defined by the World Health Organization/European Organisation for Research and Treatment of Cancer classification of cutaneous lymphomas. In some cases this disease may simulate other forms of benign or malignant solid tumours. We present a case of a 74-year-old man showing a quickly 'migrant' mass on his forehead. First skin biopsy, ultrasound and magnetic resonance images were not significant. A deeper biopsy revealed a pathology consistent with a primary cutaneous diffuse large B-cell lymphoma leg type. The patient was successfully treated with only local radiotherapy (total dose: 32.4 Gy). At 1-year follow-up there were no recurrences. To the best of our knowledge, this is the second case of a primary cutaneous diffuse large B-cell lymphoma leg type developed as a quickly 'migrant' lesion. In contrast with the first report, our case developed in a non-leg site. From these two cases, we should bear in mind that aggressive and quickly migrant cutaneous or subcutaneous masses might mask a lymphomatous disease.

Automated skin biopsy histopathological image annotation using multi-instance representation and learning.

With digitisation and the development of computer-aided diagnosis, histopathological image analysis has attracted considerable interest in recent years. In this article, we address the problem of the automated annotation of skin biopsy images, a special type of histopathological image analysis. In contrast to previous well-studied methods in histopathology, we propose a novel annotation method based on a multi-instance learning framework. The proposed framework first represents each skin biopsy image as a multi-instance sample using a graph cutting method, decomposing the image to a set of visually disjoint regions. Then, we construct two classification models using multi-instance learning algorithms, among which one provides determinate results and the other calculates a posterior probability. We evaluate the proposed annotation framework using a real dataset containing 6691 skin biopsy images, with 15 properties as target annotation terms. The results indicate that the proposed method is effective and medically acceptable.

R58: Stabilisation de CDC25B : sortie de mitose anormale et instabilité génétique.

Pathologists analyze biopsy material at both the cellular and structural level to determine diagnosis and cancer stage. Mitotic figures are surrogate biomarkers of cellular proliferation that can provide prognostic information; thus, their precise detection is an important factor for clinical care. Convolutional Neural Networks (CNNs) have shown remarkable performance on several recognition tasks. Utilizing CNNs for mitosis classification may aid pathologists to improve the detection accuracy.We studied two state-of-the-art CNN-based models, ESPNet and DenseNet, for mitosis classification on six whole slide images of skin biopsies and compared their quantitative performance in terms of sensitivity, specificity, and F-score. We used raw RGB images of mitosis and non-mitosis samples with their corresponding labels as training input. In order to compare with other work, we studied the performance of these classifiers and two other architectures, ResNet and ShuffleNet, on the publicly available MITOS breast biopsy dataset and compared the performance of all four in terms of precision, recall, and F-score (which are standard for this data set), architecture, training time and inference time.The ESPNet and DenseNet results on our primary melanoma dataset had a sensitivity of 0.976 and 0.968, and a specificity of 0.987 and 0.995, respectively, with F-scores of .968 and .976, respectively. On the MITOS dataset, ESPNet and DenseNet showed a sensitivity of 0.866 and 0.916, and a specificity of 0.973 and 0.980, respectively. The MITOS results using DenseNet had a precision of 0.939, recall of 0.916, and F-score of 0.927. The best published result on MITOS (Saha et al. 2018) reported precision of 0.92, recall of 0.88, and F-score of 0.90. In our architecture comparisons on MITOS, we found that DenseNet beats the others in terms of F-Score (DenseNet 0.927, ESPNet 0.890, ResNet 0.865, ShuffleNet 0.847) and especially Recall (DenseNet 0.916, ESPNet 0.866, ResNet 0.807, ShuffleNet 0.753), while ResNet and ESPNet have much faster inference times (ResNet 6 s, ESPNet 8 s, DenseNet 31 s). ResNet is faster than ESPNet, but ESPNet has a higher F-Score and Recall than ResNet, making it a good compromise solution.We studied several state-of-the-art CNNs for detecting mitotic figures in whole slide biopsy images. We evaluated two CNNs on a melanoma cancer dataset and then compared four CNNs on a public breast cancer data set, using the same methodology on both. Our methodology and architecture for mitosis finding in both melanoma and breast cancer whole slide images has been thoroughly tested and is likely to be useful for finding mitoses in any whole slide biopsy images.

Spectroscopic analysis of keratin endogenous signal for skin multiphoton microscopy

We recorded one-photon excited fluorescence (1PEF) and two-photon excited fluorescence (2PEF) spectra of purified keratin from human epidermis, and determined the action cross section of this endogenous chromophore. We used this spectroscopic analysis to analyse multiphoton images of skin biopsies and assign the intrinsic fluorescence signals in the epidermis. We observed a good agreement between in situ and in vitro 2PEF spectra of keratin. This study provides a comprehensive characterization of the 2PEF signal of the keratins from the epidermis, and will be of practical interest for multiphoton imaging of the skin.