Abstract 303: Cell Slider: Using crowd sourcing for the scoring of molecular pathology

Abstract

Abstract The advent of tumour microarrays has increased the throughput of pathology sample preparation but has led to a new bottleneck at the analysis stage as many thousands of samples need to be scored by an expert pathologist. We launched Cell Slider (www.clicktocure.net), the world's first Citizen Science project in cancer research, in Autumn 2012. Cell SliderTM aims to speed up the analysis of tumour microarray images through crowd sourcing. Citizen Scientists have now generated over 1.7 million scores of tumour images from the Breast Cancer Association Consortium image database. In order to evaluate the validity of Citizen Scientist scoring we compared the scores of Citizen Scientists with those of an experienced pathologist for 2,575 tissue cores from the SEARCH study that have been stained for expression of the estrogen receptor. Each image of a 0.6mm tissue core stained for ER was divided into 12 sub-images, each of which was scored by five citizen scientists. A weighted average of the 60 (12x5) scores was used to generate a single citizen scientist score for each tissue core. The average number of cancer cells observed by Citizen Scientists was used to indicate whether or not the core contained cancer. The sensitivity and specificity then depends on the threshold used for positivity. Of 2,575 cores on the TMA the pathologist identified 1,792 as having cancer tissue. The kappa statistic for agreement between pathologist and Citizen Scientist was 0.72. Using a threshold to give a sensitivity of 95 per cent, the specificity was 72 per cent. We estimated the accuracy of assignment of tumour ER status using the 1,703 cores where both pathologist and Citizen Scientists identified cancer in the TMA core. A pseudo-Allred score on a continuous scale was calculated from the individual Citizen Scientist scores for the proportion of positively stained cells and the intensity of positively stained cells. The sensitivity and specificity of ER status depends on the threshold pseudo-Allred score used to determine ER positivity. The kappa statistic for agreement between pathologist and Citizen Scientist was 0.84. Using a threshold to give a sensitivity of 95 per cent, the specificity was 90 per cent. We have shown that crowd sourcing can be used to score breast cancer TMAs stained by immunohistochemistry with a reasonable degree of accuracy. Further analysis are underway to determine whether accuracy can be improved using difference weighting schemes and to evaluate the accuracy of scoring of typical cytoplasmic and membrane molecular markers. Citation Format: Paul D. Pharoah. Cell Slider: Using crowd sourcing for the scoring of molecular pathology. [abstract]. In: Proceedings of the 105th Annual Meeting of the American Association for Cancer Research; 2014 Apr 5-9; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2014;74(19 Suppl):Abstract nr 303. doi:10.1158/1538-7445.AM2014-303