A Pilot Benchmarking Study of Deep Neural Network Performance on Low Magnification Pathology Region-of-Interest Images

Abstract

Deep Neural Networks (DNNs) have successfully demonstrated superior overall performance in many image classification and recognition tasks on Hematoxylin and Eosin stain (H&E) histology images. Reported studies typically utilize high quality (20x or 40x) Whole Slide Images (WSIs) to deliver optimal performance. However, it remains uncertain how well DNNs can perform on lower quality Region of Interest (ROI) histology images in real-life scenarios. The NCI Patient Derived Models Repository (PDMR) database hosts a catalog of low magnification (4x) ROIs of tissue histology images across a total of 60 cancer models, providing an ideal test case for evaluating DNNs performance in real-life scenarios. Using five pre-trained models, we have benchmarked the NCI PDMR database ROIs on a selected set of popular DNN classifiers. Overall, on the binary carcinoma vs. sarcoma classification test, we have reached 89.57% accuracy on 4x ROIs using our downsizing models and 84.18% accuracy on 4x ROIs using our patch-based models. On the multi-class carcinoma classification test, we have reached 72.06% top-2 accuracy on 4x ROIs using our downsizing models and 78.07% top-2 accuracy on 4x ROIs using our patch-based models. Given that pathologist accuracies hover around 85% [26] [27], our models were comparable in performance. With such accuracies, we can utilize our DNNs to perform crucial tele-pathological tasks in underdeveloped countries and rural areas, enabling scientists to take a cell phone picture and feed that image into a battery powered small computer for rapid screenings on the field.