Sensitivity analysis in digital pathology: Handling large number of parameters with compute expensive workflows

Jeremias Gomes,Willian Barreiros,Tahsin Kurc,Alba C.M.A Melo,Jun Kong,Joel H Saltz,George Teodoro

doi:10.1016/j.compbiomed.2019.03.006

Abstract

Digital pathology imaging enables valuable quantitative characterizations of tissue state at the sub-cellular level. While there is a growing set of methods for analysis of whole slide tissue images, many of them are sensitive to changes in input parameters. Evaluating how analysis results are affected by variations in input parameters is important for the development of robust methods. Executing algorithm sensitivity analyses by systematically varying input parameters is an expensive task because a single evaluation run with a moderate number of tissue images may take hours or days. Our work investigates the use of Surrogate Models (SMs) along with parallel execution to speed up parameter sensitivity analysis (SA). This approach significantly reduces the SA cost, because the SM execution is inexpensive. The evaluation of several SM strategies with two image segmentation workflows demonstrates that a SA study with SMs attains results close to a SA with real application runs (mean absolute error lower than 0.022), while the SM accelerates the SA execution by 51 × . We also show that, although the number of parameters in the example workflows is high, most of the uncertainty can be associated with a few parameters. In order to identify the impact of variations in segmentation results to downstream analyses, we carried out a survival analysis with 387 Lung Squamous Cell Carcinoma cases. This analysis was repeated using 3 values for the most significant parameters identified by the SA for the two segmentation algorithms; about 600 million cell nuclei were segmented per run. The results show that significance of the survival correlations of patient groups, assessed by a logrank test, are strongly affected by the segmentation parameter changes. This indicates that sensitivity analysis is an important tool for evaluating the stability of conclusions from image analyses.

Highlights

Analysis of whole slide tissue images (WSIs) is an important component of biomedical imaging studies
We have developed an optimization in Region Templates (RT) for Sensitivity Analysis (SA) studies, called Reuse Tree Merging (RTM) strategy, that enables reuse of repeated computations from multiple application runs [39]
We carried out small scale experiments after the generation of the training dataset on a local machine with an Intel (R) Core (TM) E5–2640 CPU with 2.60GHz, 64GB of RAM

Summary

Introduction

Analysis of whole slide tissue images (WSIs) is an important component of biomedical imaging studies. Quantitative data computed from WSIs and their correlation with other sources of information, such as clinical and molecular data, can assist in understanding disease mechanisms as well as in disease grading and classification [1, 2, 3, 4, 5, 6, 7]. While there are several benefits in using WSIs, many challenges, such as accuracy and robustness of analysis workflows and high computation costs, have to be addressed. One of the methodology challenges is the fact that analysis workflows are often sensitive to input parameters. Sensitivity Analysis (SA) is defined as the process of quantifying the inherent variability observed in results from data analyses when the values of input parameters are varied. A SA study can be used in image analysis to test robustness of analysis results, to understand correlations among input parameters and analysis output, to guide parameter tuning, and to simplify an analysis workflow. A SA with a large number of WSIs, is a computationally very demanding process, which limits its use in practice

Methods

Results

Discussion

Conclusion