CS-Net: Instance-aware cellular segmentation with hierarchical dimension-decomposed convolutions and slice-attentive learning

Abstract

Cellular segmentation in kinds of microscopy images is a fundamental prerequisite in the workflow of many biomedical applications. However, accurate segmentation of plenty of cellular instances faces several challenges, including instance-size imbalance, clustered cellular instances, fuzzy boundaries, and cell types’ heterogeneity. Moreover, both 2D and 3D data should be processed with scalability. To address these challenges, we present a lightweight slice-wise CS-Net building on novel hierarchical dimension-decomposed (HDD) convolutions and a novel instance-aware loss for both 2D and 3D microscopy image segmentation. To capture inter-slice contexts in 3D data, we exploit both 2.5D input and 2.5D supervision, and introduce CS-Net (2.5D++) for 3D segmentation tasks. Specifically, we devise a slice-aware encoder to extract diverse and multiscale contexts from multi-slice input and a slice-attentive decoder to take advantage of slice-wise 2.5D supervision. To separate clustered instances, we augment the segmentation task with an edge prediction task, the output of which is used to separate touching instances. Extensive experiments on 3D multi-tissue electron microscopy (EM) data and 2D histology images demonstrate that our 2D and 2.5D methods achieve state-of-the-art performance with scalability to diverse data. The comparative results with top-performing lightweight models also indicate that the proposed model shows a better balance of segmentation performance and computational complexity.