An automated algorithm for cell-level FISH dot counting

Abstract

Fluorescence in situ hybridization (FISH) dot counting is the process of enumerating chromosomal abnormalities in interphase cell nuclei. This process is widely used in many areas of biomedical research and diagnosis. We present a generic and fully automatic algorithm for cell-level counting of FISH dots in 2-D fluorescent images. Our proposed algorithm starts by segmenting cell nuclei in DAPI stained images using a 2-D wavelet based segmentation algorithm. Nuclei segmentation is followed by FISH dot detection and counting, which consists of three main steps. First, image pre-processing where median and top-hat filters are used to clean image noise, subtract background and enhance the contrast of the FISH dots. Second, FISH dot detection using a multi-level h-minima transform approach that accounts for the varying image contrast. Third, FISH dot counting where clustered FISH dots are separated using a local maxima detection-based method followed by FISH dot size filtering based on constraints to account for large connected components of tightly-clustered dots. To quantitatively assess the performance of our proposed FISH dot counting algorithm, automatic counting results were compared to manual counts of 880 cells selected from 19 invasive ductal breast carcinoma samples exhibiting varying degrees of Human Epidermal Growth Factor Receptor 2 (HER2) expression. Cell-level dot counting accuracy was assessed using two metrics: cell classification agreement and dot-counting match. Our automatic results gave an overall cell-by-cell classification agreement of 88% and an overall accuracy of 81%.