High-resolution Histology Images Research Articles

Abstract 7424: iStarTLS: Advanced detection and phenotyping of tertiary lymphoid structures

Abstract Tertiary lymphoid structures (TLSs) are clusters of immune cells formed in non-lymphoid tissues. They are often found at sites of chronic inflammation, notably within the invasive margins and the core of various solid tumors. TLSs are pivotal in mediating anti-tumor immunity. However, our understanding of TLSs in large/complex tissue contexts remains incomplete due to the lack of computational tools to effectively detect and phenotype TLSs. Recent advances in spatially resolved transcriptomics (SRT) present a broader spectrum of analytical possibilities for investigating the spatial phenotypic heterogeneity of TLSs and their interaction with stromal and cancer cells. Here, we present iStarTLS (Inferring Super-resolution Tissue ARchitecture for TLSs), a computational toolkit designed to process SRT data for TLS detection and phenotyping and showcase its performance on breast, bladder, and lung cancer samples. By effectively integrating spatial gene expression data with state-of-the-art machine learning techniques, we can substantially enhance our capabilities in TLS detection and comprehensive phenotyping. iStarTLS starts by enhancing the spatial resolution of spot-level gene expression data to near-single-cell resolution by leveraging high-resolution information provided by paired histology images. To detect TLSs and infer their cellular composition, we developed a TLS signature. Based on the high-resolution gene expression measurements and a curated reference panel of cell type-specific markers, we score cell type-specific gene signatures to obtain a cell type probability map across the whole tissue section. This map gives rise to a segmentation of key cell type components of TLSs, enabling the spatial mapping and colocalization of different cell types. Moreover, such an approach would allow us to infer the phenotypic states of cells within the TLSs, assess their cellular compositions, and discern their cellular organization in large, spatially heterogeneous tissues at a near-single-cell resolution. Notably, in conjunction with nuclei segmentation of high-resolution histology images, iStarTLS precisely maps high endothelial venules (HEVs), a key structure within TLSs often overlooked by previous studies. iStarTLS paves the way for uncovering novel mechanisms of immune-tumor interactions and designing personalized therapies targeting specific cellular components or states within TLSs. Citation Format: Kyung Serk Cho, Jiahui Jiang, Daiwei Zhang, Yunhe Liu, Jianfeng Chen, Rossana L. Segura, Xinmiao Yan, Guangsheng Pei, Luisa M. Soto, Yanshuo Chu, Ansam F. Sinjab, Cassian Yee, Scott Kopetz, Anirban Maitra, Andrew Futreal, Alexander Lazar, Amir A. Jazaeri, Humam Kadara, Jianjun Gao, Mingyao Li, Linghua Wang. iStarTLS: Advanced detection and phenotyping of tertiary lymphoid structures [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7424.

Improving Breast Tumor Multi-Classification from High-Resolution Histological Images with the Integration of Feature Space Data Augmentation

To support pathologists in breast tumor diagnosis, deep learning plays a crucial role in the development of histological whole slide image (WSI) classification methods. However, automatic classification is challenging due to the high-resolution data and the scarcity of representative training data. To tackle these limitations, we propose a deep learning-based breast tumor gigapixel histological image multi-classifier integrated with a high-resolution data augmentation model to process the entire slide by exploring its local and global information and generating its different synthetic versions. The key idea is to perform the classification and augmentation in feature latent space, reducing the computational cost while preserving the class label of the input. We adopt a deep learning-based multi-classification method and evaluate the contribution given by a conditional generative adversarial network-based data augmentation model on the classifier’s performance for three tumor classes in the BRIGHT Challenge dataset. The proposed method has allowed us to achieve an average F1 equal to 69.5, considering only the WSI dataset of the Challenge. The results are comparable to those obtained by the Challenge winning method (71.6), also trained on the annotated tumor region dataset of the Challenge.

Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology.

Spatial transcriptomics (ST) has demonstrated enormous potential for generating intricate molecular maps of cells within tissues. Here we present iStar, a method based on hierarchical image feature extraction that integrates ST data and high-resolution histology images to predict spatial gene expression with super-resolution. Our method enhances gene expression resolution to near-single-cell levels in ST and enables gene expression prediction in tissue sections where only histology images are available.

Ultrahigh Resolution OCT Markers of Normal Aging and Early Age-related Macular Degeneration.

Ultrahigh resolution spectral domain-OCT (UHR SD-OCT) enables invivo visualization of micrometric structural markers which differentially associate with normal aging versus age-related macular degeneration (AMD). This study explores the hypothesis that UHR SD-OCT can detect and quantify sub-retinal pigment epithelium (RPE) deposits in early AMD, separating AMD pathology from normal aging. Prospective cross-sectional study. A total of 53 nonexudative (dry) AMD eyes from 39 patients, and 63 normal eyes from 39 subjects. Clinical UHR SD-OCT scans were performed using a high-density protocol. Exemplary high-resolution histology and transmission electron microscopy images were obtained from archive donor eyes. Three trained readers evaluated and labeled outer retina morphological features, including the appearance of a hyporeflective split within the RPE-RPE basal lamina (RPE-BL)-Bruch's membrane (BrM) complex on UHR brightness (B)-scans. A semi-automatic segmentation algorithm measured the thickness of the RPE-BL-BrM split/hyporeflective band. Qualitative description of outer retinal morphological changes on UHR SD-OCT B-scans; the proportion of the RPE-BL-BrM complex with visible split (%) and the thickness of the resulting hyporeflective band (μm). In young normal eyes, UHR SD-OCT consistently revealed an RPE-BL-BrM split/hyporeflective band. Its visibility and thickness were less in eyes of advanced age. However, the split/hyporeflective band was again visible in early AMD eyes. Both qualitative reading and quantitative thickness measurements showed significantly elevated visibility and thickness of the RPE-BL-BrM split/hyporeflective in early AMD eyes compared to age-matched controls. Our imaging results strongly support the hypothesis that appearance of the RPE-BL-BrM split/hyporeflective band in older subjects is dominated by the BL deposit, an indicator of early AMD well known from histology. Ultrahigh resolution SD-OCT can be used to investigate physiological aging as well as early AMD pathology in clinical imaging studies. Developing quantifiable markers associated with disease pathogenesis and progression can facilitate drug discovery, as well as reduce clinical trial times. Proprietary or commercial disclosure may be found after the references.

A Standardized Protocol for the Safe Retrieval of Infectious Postmortem Human Brain for Studying Whole-Brain Pathology.

We describe a safe and standardized perfusion protocol for studying brain pathology in high-risk autopsies using a custom-designed low-cost infection containment chamber and high-resolution histology. The output quality was studied using the histological data from the whole cerebellum and brain stem processed using a high-resolution cryohistology pipeline at 0.5 μm per pixel, in-plane resolution with serial sections at 20-μm thickness. To understand the pathophysiology of highly infectious diseases, it is necessary to have a safe and cost-effective method of performing high-risk autopsies and a standardized perfusion protocol for preparing high-quality tissues. Using the low-cost infection containment chamber, we detail the cranial autopsy protocol and ex situ perfusion-fixation of 4 highly infectious adult human brains. The digitized high-resolution histology images of the Nissl-stained series reveal that most of the sections were free of processing artifacts, such as fixation damage, freezing artifacts, and osmotic shock, at the macrocellular and microcellular level. The quality of our protocol was also tested with the highly sensitive immunohistochemistry staining for specific protein markers. Our protocol provides a safe and effective method in high-risk autopsies that allows for the evaluation of pathogen-host interaction, the underlying pathophysiology, and the extent of the infection across the whole brain at microscopic resolutions.

Computational homogenization of histological microstructures in human prostate tissue: Heterogeneity, anisotropy and tension-compression asymmetry.

Human prostatic tissue exhibits complex mechanical behaviour due to its multiphasic, heterogeneous nature, with hierarchical microstructures involving epithelial compartments, acinar lumens and stromal tissue all interconnected in complex networks. This study aims to establish a computational homogenization framework for quantifying the mechanical behaviour of prostate tissue, considering its multiphasic heterogeneous microstructures and the mechanical characteristics of tissue constituents. Representative tissue microstructure models were reconstructed from high-resolution histology images. Parametric studies on the mechanical properties of the tissue constituents, particularly the fibre-reinforced hyper-elasticity of the stromal tissue, were carried out to investigate their effects on the apparent tissue properties. These were then benchmarked against the experimental data reported in literature. Results showed significant anisotropy, both structural and mechanical, and tension-compression asymmetry of the apparent behaviours of the prostatic tissue. Strong correlation with the key microstructural indices such as area fractions of tissue constituents and the tissue fabric tensor was also observed. The correlation between the stromal tissue orientation and the principal directions of the apparent properties suggested an essential role of stromal tissue in determining the directions of anisotropy and the compression-tension asymmetry characteristics in normal human prostatic tissue. This work presented a homogenization and histology-based computational approach to characterize the apparent mechanical behaviours of human prostatic or other similar glandular tissues, with the ultimate aim of assessing how pathological conditions (e.g., prostate cancer and benign prostatic hyperplasia) could affect the tissue mechanical properties in a future study.

Integrating multi-modal information to detect spatial domains of spatial transcriptomics by graph attention network

Recent advances in spatially resolved transcriptomic technologies have enabled unprecedented opportunities to elucidate tissue architecture and function in situ. Spatial transcriptomics can provide multimodal and complementary information simultaneously, including gene expression profiles, spatial locations, and histology images. However, most existing methods have limitations in efficiently utilizing spatial information and matched high-resolution histology images. To fully leverage the multi-modal information, we propose a SPAtially embedded Deep Attentional graph Clustering (SpaDAC) method to identify spatial domains while reconstructing denoised gene expression profiles. This method can efficiently learn the low-dimensional embeddings for spatial transcriptomics data by constructing multi-view graph modules to capture both spatial location connectives and morphological connectives. Benchmark results demonstrate that SpaDAC outperforms other algorithms on several recent spatial transcriptomics datasets. SpaDAC is a valuable tool for spatial domain detection, facilitating the comprehension of tissue architecture and cellular microenvironment. The source code of SpaDAC is freely available at Github (https://github.com/huoyuying/SpaDAC.git).

Imaging of lumpectomy surface with large field-of-view confocal laser scanning microscopy ‘Histolog® scanner’ for breast margin assessment in comparison with conventional specimen radiography

PurposeThe Histolog® Scanner (SamanTree Medical SA, Lausanne, Switzerland) is a large field-of-view confocal laser scanning microscope designed to allow intraoperative margin assessment by the production of histological images ready for assessment in the operating room. We evaluated the feasibility and the performance of the Histolog® Scanner (HS) to correctly identify infiltrated margins in clinical practice of lumpectomy specimens. It was extrapolated if the utilization of the HS has the potential to reduce infiltrated margins and therefore reduce re-operation rates in patients undergoing breast conserving surgery (BCS) due to a primarily diagnosed breast cancer including ductal carcinoma in situ. MethodsThis is a single-center, prospective, non-interventional, diagnostic pilot study including 50 consecutive patients receiving BCS. The complete surface of the specimen was scanned using the HS intraoperatively. The surgery and the intraoperative margin assessment of the specimen was performed according to the clinical routine consisting of conventional specimen radiography as well as the clinical impression of the surgeon. Three surgeons and an experienced pathologist assessed the scans produced by the HS for cancer cells on the surface. The potential of the HS to correctly identify involved margins was compared to the results of the conventional specimen radiography alone as well as the clinical routine. The histopathological report served as the gold standard. Results50 specimens corresponding to 300 surfaces were scanned by the HS. The mean sensitivity of the surgeons to identify involved margins with the HS was 37.5% ± 5.6%, the specificity was 75.2% ± 13.0%. The assessment of resection margins by the pathologist resulted in a sensitivity of 37.5% and a specificity of 81.0%, while the local clinical routine resulted in a sensitivity of 37.5% and a specificity of 78.2%. ConclusionAcquisition of high-resolution histological images using the HS was feasible in clinical practice. Sensitivity and specificity were comparable to clinical routine. With more specific training and experience on image interpretation and acquisition, the HS may have the potential to enable more accuracy in the margin assessment of BCS specimens.

Patient-specific voxel-level dose prescription for prostate cancer radiotherapy considering tumor cell density and grade distribution.

In prostate radiation therapy, recent studies have indicated a benefit in increasing the dose to intraprostatic lesions (IPL) compared with standard whole gland radiation therapy. Such approaches typically aim to deliver a target dose to the IPL(s) with no deliberate effort to modulate the dose within the IPL. Prostate cancers demonstrate intra-tumor heterogeneity and hence it is hypothesized that further gains in the optimal delivery of radiation therapy can be achieved through modulation of the dose distribution within the tumor. To account for tumor heterogeneity, biologically targeted radiation therapy (BiRT) aims to utilize a voxel-wise approach to IPL dose prescription by incorporating knowledge of the spatial distribution of tumor characteristics. The aim of this study was to develop a workflow for generating voxel-wise optimal dose prescriptions that maximize patient tumor control probability (TCP), and evaluate the feasibility and benefits of applying this workflow on a cohort of 62 prostate cancer patients. The source data for this proof-of-concept study included high resolution histology images annotated with tumor location and grade. Image processing techniques were used to compute voxel-level cell density distribution maps. An absolute tumor cell distribution was calculated via linearly scaling according to published estimated tumor cell numbers. For the IPLs of each patient, optimal dose prescriptions were obtained via three alternative methods for redistribution of IPL boost doses according to maximization of TCP. The radiosensitivity uncertainties were considered using a truncated log-normally distributed linear radiosensitivity parameter ( ) and compared with Gleason pattern (GP) dependent radiosensitivity parameters that were derived based on previously published methods. An ensemble machine learning method was implemented to identify patient-specific features that predict the TCP improvement resulting from dose redistribution relative to a uniform dose distribution. The Gleason pattern-dependent radiosensitivity parameters were calculated for 20 published prostate cancer ratios. Optimal voxel-level dose prescriptions were generated for all 62 PCa patients. For all dose redistribution scenarios, the optimal dose distribution always shows a higher (or equivalent) TCP level than the uniform dose distribution. The applied random forest regressor could predict patient-specific TCP improvement with low root mean square error (≤1.5%) by using total tumor number, volume of IPLs and the standard deviation of tumor cell number among all voxels. Biologically-optimized redistribution of a boost dose can yield TCP improvement relative to a uniform-boost dose distribution. Patient-specific tumor characteristics can be used to predict the likelihood of benefit from a redistribution approach for the individual patient.

Super-resolution segmentation network for inner-ear tissue segmentation.

Cochlear implants (CIs) are considered the standard-of-care treatment for profound sensory-based hearing loss. Several groups have proposed computational models of the cochlea in order to study the neural activation patterns in response to CI stimulation. However, most of the current implementations either rely on high-resolution histological images that cannot be customized for CI users or CT images that lack the spatial resolution to show cochlear structures. In this work, we propose to use a deep learning-based method to obtain μCT level tissue labels using patient CT images. Experiments showed that the proposed super-resolution segmentation architecture achieved very good performance on the inner-ear tissue segmentation. Our best-performing model (0.871) outperformed the UNet (0.746), VNet (0.853), nnUNet (0.861), TransUNet (0.848), and SRGAN (0.780) in terms of mean dice score.

VistoSeg: Processing utilities for high-resolution images for spatially resolved transcriptomics data.

Spatially resolved transcriptomics (SRT) is a growing field that links gene expression to anatomical context. SRT approaches that use next-generation sequencing (NGS) combine RNA sequencing with histological or fluorescent imaging to generate spatial maps of gene expression in intact tissue sections. These technologies directly couple gene expression measurements with high-resolution histological or immunofluorescent images that contain rich morphological information about the tissue under study. While broad access to NGS-based spatial transcriptomic technology is now commercially available through the Visium platform from the vendor 10× Genomics, computational tools for extracting image-derived metrics for integration with gene expression data remain limited. We developed VistoSeg as a MATLAB pipeline to process, analyze and interactively visualize the high-resolution images generated in the Visium platform. VistoSeg outputs can be easily integrated with accompanying transcriptomic data to facilitate downstream analyses in common programing languages including R and Python. VistoSeg provides user-friendly tools for integrating image-derived metrics from histological and immunofluorescent images with spatially resolved gene expression data. Integration of this data enhances the ability to understand the transcriptional landscape within tissue architecture. VistoSeg is freely available at http://research.libd.org/VistoSeg/.

High-Speed Ultraviolet Photoacoustic Microscopy for Histological Imaging with Virtual-Staining assisted by Deep Learning.

Surgical margin analysis (SMA), an essential procedure to confirm the complete excision of cancerous tissue in tumor resection surgery, requires intraoperative diagnostic tools to avoid repeated surgeries due to a positive surgical margin. Recently, by taking the advantage of the high intrinsic optical absorption of DNA/RNA at 266 nm wavelength, ultraviolet photoacoustic microscopy (UV-PAM) has been developed to provide high-resolution histological images without labeling, showing great promise as an intraoperative tool for SMA. To enable the development of UV-PAM for SMA, here, a high-speed and open-top UV-PAM system is presented, which can be operated similarly to conventional optical microscopies. The UV-PAM system provides a high lateral resolution of 1.2 µm, and a high imaging speed of 55 kHz A-line rate with one-axis galvanometer mirror scanning. Moreover, to ensure UV-PAM images can be easily interpreted by pathologists without additional training, the original grayscale UV-PAM images are virtually stained by a deep-learning algorithm to mimic the standard hematoxylin- and eosin-stained images, enabling training-free histological analysis. Mouse brain slice imaging is performed to demonstrate the high performance of the open-top UV-PAM system, illustrating its great potential for SMA applications.

Reduced and stable feature sets selection with random forest for neurons segmentation in histological images of macaque brain

In preclinical research, histology images are produced using powerful optical microscopes to digitize entire sections at cell scale. Quantification of stained tissue relies on machine learning driven segmentation. However, such methods require multiple additional information, or features, which are increasing the quantity of data to process. As a result, the quantity of features to deal with represents a drawback to process large series or massive histological images rapidly in a robust manner. Existing feature selection methods can reduce the amount of required information but the selected subsets lack reproducibility. We propose a novel methodology operating on high performance computing (HPC) infrastructures and aiming at finding small and stable sets of features for fast and robust segmentation of high-resolution histological images. This selection has two steps: (1) selection at features families scale (an intermediate pool of features, between spaces and individual features) and (2) feature selection performed on pre-selected features families. We show that the selected sets of features are stables for two different neuron staining. In order to test different configurations, one of these dataset is a mono-subject dataset and the other is a multi-subjects dataset to test different configurations. Furthermore, the feature selection results in a significant reduction of computation time and memory cost. This methodology will allow exhaustive histological studies at a high-resolution scale on HPC infrastructures for both preclinical and clinical research.

A convolutional neural network for common coordinate registration of high-resolution histology images.

MotivationRegistration of histology images from multiple sources is a pressing problem in large-scale studies of spatial -omics data. Researchers often perform ‘common coordinate registration’, akin to segmentation, in which samples are partitioned based on tissue type to allow for quantitative comparison of similar regions across samples. Accuracy in such registration requires both high image resolution and global awareness, which mark a difficult balancing act for contemporary deep learning architectures.ResultsWe present a novel convolutional neural network (CNN) architecture that combines (i) a local classification CNN that extracts features from image patches sampled sparsely across the tissue surface and (ii) a global segmentation CNN that operates on these extracted features. This hybrid network can be trained in an end-to-end manner, and we demonstrate its relative merits over competing approaches on a reference histology dataset as well as two published spatial transcriptomics datasets. We believe that this paradigm will greatly enhance our ability to process spatial -omics data, and has general purpose applications for the processing of high-resolution histology images on commercially available GPUs.Availability and implementationAll code is publicly available at https://github.com/flatironinstitute/st_gridnet.Supplementary information Supplementary data are available at Bioinformatics online.

LM-GlycomeAtlas Ver. 2.0: An Integrated Visualization for Lectin Microarray-based Mouse Tissue Glycome Mapping Data with Lectin Histochemistry

Laser microdissection-assisted lectin microarray has been used to obtain quantitative and qualitative information on glycans on proteins expressed in microscopic regions of formalin-fixed paraffin-embedded tissue sections. For the effective visualization of this "tissue glycome mapping" data, a novel online tool, LM-GlycomeAtlas (https://glycosmos.org/lm_glycomeatlas/index), was launched in the freely available glycoscience portal, the GlyCosmos Portal (https://glycosmos.org). In LM-GlycomeAtlas Version 1.0, nine tissues from normal mice were used to provide one data set of glycomic profiles. Here we introduce an updated version of LM-GlycomeAtlas, which includes more spatial information. We designed it to deposit multiple data sets of glycomic profiles with high-resolution histological images, which included staining images with multiple lectins on the array. The additionally implemented interfaces allow users to display multiple histological images of interest (e.g., diseased and normal mice), thereby facilitating the evaluation of tissue glycomic profiling and glyco-pathological analysis. Using these updated interfaces, 451 glycomic profiling data and 42 histological images obtained from 14 tissues of normal and diseased mice were successfully visualized. By easy integration with other tools for glycoproteomic data and protein glycosylation machinery, LM-GlycomeAtlas will be one of the most valuable open resources that contribute to both glycoscience and proteomics communities.

Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization.

High-quality whole slide scanners used for animal and human pathology scanning are expensive and can produce massive datasets, which limits the access to and adoption of this technique. As a potential solution to these challenges, we present a deep learning-based approach making use of single image super-resolution (SISR) to reconstruct high-resolution histology images from low-resolution inputs. Such low-resolution images can easily be shared, require less storage, and can be acquired quickly using widely available low-cost slide scanners. The network consists of multi-scale fully convolutional networks capable of capturing hierarchical features. Conditional generative adversarial loss is incorporated to penalize blurriness in the output images. The network is trained using a progressive strategy where the scaling factor is sampled from a normal distribution with an increasing mean. The results are evaluated with quantitative metrics and are used in a clinical histopathology diagnosis procedure which shows that the SISR framework can be used to reconstruct high-resolution images with clinical level quality. We further propose a self-supervised color normalization method that can remove staining variation artifacts. Quantitative evaluations show that the SISR framework can generalize well on unseen data collected from other patient tissue cohorts by incorporating the color normalization method.

DeepHistReg: Unsupervised Deep Learning Registration Framework for Differently Stained Histology Samples

Background and objectiveThe use of several stains during histology sample preparation can be useful for fusing complementary information about different tissue structures. It reveals distinct tissue properties that combined may be useful for grading, classification, or 3-D reconstruction. Nevertheless, since the slide preparation is different for each stain and the procedure uses consecutive slices, the tissue undergoes complex and possibly large deformations. Therefore, a nonrigid registration is required before further processing. The nonrigid registration of differently stained histology images is a challenging task because: (i) the registration must be fully automatic, (ii) the histology images are extremely high-resolution, (iii) the registration should be as fast as possible, (iv) there are significant differences in the tissue appearance, and (v) there are not many unique features due to a repetitive texture. MethodsIn this article, we propose a deep learning-based solution to the histology registration. We describe a registration framework dedicated to high-resolution histology images that can perform the registration in real-time. The framework consists of an automatic background segmentation, iterative initial rotation search and learning-based affine/nonrigid registration. ResultsWe evaluate our approach using an open dataset provided for the Automatic Non-rigid Histological Image Registration (ANHIR) challenge organized jointly with the IEEE ISBI 2019 conference. We compare our solution to the challenge participants using a server-side evaluation tool provided by the challenge organizers. Following the challenge evaluation criteria, we use the target registration error (TRE) as the evaluation metric. Our algorithm provides registration accuracy close to the best scoring teams (median rTRE 0.19% of the image diagonal) while being significantly faster (the average registration time is about 2 seconds). ConclusionsThe proposed framework provides results, in terms of the TRE, comparable to the best-performing state-of-the-art methods. However, it is significantly faster, thus potentially more useful in clinical practice where a large number of histology images are being processed. The proposed method is of particular interest to researchers requiring an accurate, real-time, nonrigid registration of high-resolution histology images for whom the processing time of traditional, iterative methods in unacceptable. We provide free access to the software implementation of the method, including training and inference code, as well as pretrained models. Since the ANHIR dataset is open, this makes the results fully and easily reproducible.

Exploring inflammatory signatures in arthritic joint biopsies with Spatial Transcriptomics

Lately it has become possible to analyze transcriptomic profiles in tissue sections with retained cellular context. We aimed to explore synovial biopsies from rheumatoid arthritis (RA) and spondyloarthritis (SpA) patients, using Spatial Transcriptomics (ST) as a proof of principle approach for unbiased mRNA studies at the site of inflammation in these chronic inflammatory diseases. Synovial tissue biopsies from affected joints were studied with ST. The transcriptome data was subjected to differential gene expression analysis (DEA), pathway analysis, immune cell type identification using Xcell analysis and validation with immunohistochemistry (IHC). The ST technology allows selective analyses on areas of interest, thus we analyzed morphologically distinct areas of mononuclear cell infiltrates. The top differentially expressed genes revealed an adaptive immune response profile and T-B cell interactions in RA, while in SpA, the profiles implicate functions associated with tissue repair. With spatially resolved gene expression data, overlaid on high-resolution histological images, we digitally portrayed pre-selected cell types in silico. The RA displayed an overrepresentation of central memory T cells, while in SpA effector memory T cells were most prominent. Consequently, ST allows for deeper understanding of cellular mechanisms and diversity in tissues from chronic inflammatory diseases.

Attention-Based Deep Neural Networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological Slides

Deep learning-based methods, such as the sliding window approach for cropped-image classification and heuristic aggregation for whole-slide inference, for analyzing histological patterns in high-resolution microscopy images have shown promising results. These approaches, however, require a laborious annotation process and are fragmented. To evaluate a novel deep learning method that uses tissue-level annotations for high-resolution histological image analysis for Barrett esophagus (BE) and esophageal adenocarcinoma detection. This diagnostic study collected deidentified high-resolution histological images (N = 379) for training a new model composed of a convolutional neural network and a grid-based attention network. Histological images of patients who underwent endoscopic esophagus and gastroesophageal junction mucosal biopsy between January 1, 2016, and December 31, 2018, at Dartmouth-Hitchcock Medical Center (Lebanon, New Hampshire) were collected. The model was evaluated on an independent testing set of 123 histological images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma. Performance of this model was measured and compared with that of the current state-of-the-art sliding window approach using the following standard machine learning metrics: accuracy, recall, precision, and F1 score. Of the independent testing set of 123 histological images, 30 (24.4%) were in the BE-no-dysplasia class, 14 (11.4%) in the BE-with-dysplasia class, 21 (17.1%) in the adenocarcinoma class, and 58 (47.2%) in the normal class. Classification accuracies of the proposed model were 0.85 (95% CI, 0.81-0.90) for the BE-no-dysplasia class, 0.89 (95% CI, 0.84-0.92) for the BE-with-dysplasia class, and 0.88 (95% CI, 0.84-0.92) for the adenocarcinoma class. The proposed model achieved a mean accuracy of 0.83 (95% CI, 0.80-0.86) and marginally outperformed the sliding window approach on the same testing set. The F1 scores of the attention-based model were at least 8% higher for each class compared with the sliding window approach: 0.68 (95% CI, 0.61-0.75) vs 0.61 (95% CI, 0.53-0.68) for the normal class, 0.72 (95% CI, 0.63-0.80) vs 0.58 (95% CI, 0.45-0.69) for the BE-no-dysplasia class, 0.30 (95% CI, 0.11-0.48) vs 0.22 (95% CI, 0.11-0.33) for the BE-with-dysplasia class, and 0.67 (95% CI, 0.54-0.77) vs 0.58 (95% CI, 0.44-0.70) for the adenocarcinoma class. However, this outperformance was not statistically significant. Results of this study suggest that the proposed attention-based deep neural network framework for BE and esophageal adenocarcinoma detection is important because it is based solely on tissue-level annotations, unlike existing methods that are based on regions of interest. This new model is expected to open avenues for applying deep learning to digital pathology.

HWDCNN: Multi-class recognition in breast histopathology with Haar wavelet decomposed image based convolution neural network

Among the predominant cancers, breast cancer is one of the main causes of cancer deaths impacting women worldwide. However, breast cancer classification is challenging due to numerous morphological and textural variations that appeared in intra-class images. Also, the direct processing of high resolution histological images is uneconomical in terms of GPU memory. In the present study, we have proposed a new approach for breast histopathological image classification that uses a deep convolution neural network (CNN) with wavelet decomposed images. The original microscopic image patches of 2048 × 1536 × 3 pixels are decomposed into 512 × 384 × 3 using 2-level Haar wavelet and subsequently used in proposed CNN model. The image decomposition step considerably reduces convolution time in deep CNNs and computational resources, without any performance downgrade. The CNN model extracts the deep features from Haar wavelet decomposed images and incorporates multi-scale discriminant features for precise prognostication of class labels. This paper also solves the demand for massive histopathology dataset by means of transfer learning and data augmentation techniques. The efficacy of proposed approach is corroborated on two publicly available breast histology datasets-(a) one provided as a part of international conference on image analysis and recognition (ICIAR 2018) grand challenge and (b) on BreakHis data. On the ICIAR 2018 validation data, our model showed an accuracy of 98.2% for both 4-class and 2-class recognition. Further, on hidden test data of the ICIAR 2018, we achieved an accuracy of 91%, outperforming existing state-of-the-art results significantly. Besides, on BreakHis dataset, the model achieved competing performance with 96.85% multi-class accuracy.