Robust hierarchical density estimation and regression for re-stained histological whole slide image co-registration.

Jun Jiang,Steven N Hart,Naresh Prodduturi,Thomas J Flotte,Nicholas B Larson,Pinaki Sarder

doi:10.1371/journal.pone.0220074

Abstract

For many disease conditions, tissue samples are colored with multiple dyes and stains to add contrast and location information for specific proteins to accurately identify and diagnose disease. This presents a computational challenge for digital pathology, as whole-slide images (WSIs) need to be properly overlaid (i.e. registered) to identify co-localized features. Traditional image registration methods sometimes fail due to the high variation of cell density and insufficient texture information in WSIs–particularly at high magnifications. In this paper, we proposed a robust image registration strategy to align re-stained WSIs precisely and efficiently. This method is applied to 30 pairs of immunohistochemical (IHC) stains and their hematoxylin and eosin (H&E) counterparts. Our approach advances the existing methods in three key ways. First, we introduce refinements to existing image registration methods. Second, we present an effective weighting strategy using kernel density estimation to mitigate registration errors. Third, we account for the linear relationship across WSI levels to improve accuracy. Our experiments show significant decreases in registration errors when matching IHC and H&E pairs, enabling subcellular-level analysis on stained and re-stained histological images. We also provide a tool to allow users to develop their own registration benchmarking experiments.

Highlights

Histological assessment is a critically important feature in the diagnosis, prognosis, and treatment of disease
In order to statistically evaluate the performance of our methods, we tested the difference in absolute registration error of each of the four methods (ECC, Fast Fourier Transform (FFT), Scale Invariant Feature Transform (SIFT), SIFTenh) using only level 0 image patches or our kernel density estimation (KDE)-weighted linear regression (Fig 6)
We proposed an approach that combines traditional rigid image registration methods into a framework that leverages the hierarchical nature of whole-slide images (WSIs)

Summary

Introduction

Histological assessment is a critically important feature in the diagnosis, prognosis, and treatment of disease. The computational cost of directly aligning gigapixel WSIs can be prohibitively high, so a smaller “image patch”-based method is often required By virtue of their smaller size, patchbased methods do not contain as much texture information to use for registration, which can lead to poor image alignments. WSIs with different stains may look quite different in hue and contrast, and different in local details, because H&E stains both nucleus and cytoplasm, while IHC only highlights the location of the target protein This can cause methods that rely on finding key points (key point matching) to fail, yielding more failures to align as too few suitable correlations in the images can be found.

Method

Experiments and results

Discussion