CellSegUNet: an improved deep segmentation model for the cell segmentation based on UNet++ and residual UNet models

Abstract

Cell nucleus segmentation is an important method that is widely used in the diagnosis and treatment of many diseases, as well as counting and identifying the cell nucleus. The main challenges when using this method are heterogeneous image intensities in the image, overlapping of cell nuclei, and noise. In order to overcome these difficulties, a hybrid segmentation model with attention block, CellSegUNet, is proposed, inspired by the advantageous points of UNet++ and Residual UNet models. With the proposed attention mechanism, semantic gaps that may occur are prevented by evaluating both horizontal and vertical features together. The serial and parallel connection of the convolutional blocks in the residual modules in the CellSegUNet model prevents data loss. Thus, features with stronger representation ability were obtained. The output layer, which is, especially proposed for the CellSegUNet model, calculated the differences between the data in each layer and the data in the input layer. The output value obtained from the layer level where the lowest value comes from constitutes the output of the whole system. At the same depth level, CellSegUNet versus UNet++ and ResUNet models were compared on Data Science Bowl (DSB), Sartorius Cell Instance Segmentation (SCIS), and Blood Cell Segmentation (BCS) datasets. With the CellSegUNet model, accuracy, dice, and jaccard metrics were obtained as 0.980, 0.970, 0.959 for the DSB dataset, 0.931, 0.957, 0.829 for the SCIS dataset and 0.976, 0.971, 0.927 for the BCS dataset, respectively. As a result, it is predicted that the proposed model can provide solutions to different segmentation problems.