MACD R-CNN: An Abnormal Cell Nucleus Detection Method

Abstract

The detection of abnormal cell nuclei is a key technique of the cytopathic automatic screening system, which directly determines the performance of the system. Although the Mask R-CNN which combines target detection and semantic segmentation has achieved good performance in general target detection tasks, the performance in abnormal cell detection is still unsatisfactory. To solve this problem, we design a new deep neural network for abnormal cell detection based on the Mask R-CNN, named mask abnormal cell detection R-CNN (MACD R-CNN). First, in the classification branch of Mask R-CNN, it generates the same size of feature maps from different size of RoIs as the input. The nuclei in this part of the feature maps will be deformed to varying degrees. We design a fixed proposal module to generate fixed-sized feature maps of nuclei, which allows the new information of nucleus is used for classification. Then we use the attention mechanism to merge the original RoI and Fixed RoI features. Finally, we increase the depth of the convolution layer to further improve the accuracy of cell classification. Experiments show that the MACD R-CNN can effectively improve the performance of abnormal cell detection.