Abstract 2306: A deep learning framework for automated segmentation and analysis of 3D breast cancer spheroid invasion dynamics

Abstract

Abstract Invasion of neighboring tissues is a cardinal feature of malignancy, notably observed in aggressive cancers like breast cancer, where it can lead to significant morbidity. To explore cancer invasion dynamics, the implementation of a three-dimensional (3D) tumor spheroid invasion assay offers a swift approach to mimicking a tumor micro-region or micro-metastasis. Differential Interference Contrast (DIC) time-lapse imaging was chosen for its fluorescence-free and non-destructive advantages in capturing live spheroid movement. However, the subsequent analysis posed challenges, including out-of-focus cells and a voluminous dataset with diverse microscope planes and time points. Our study introduces a segmentation framework featuring in-focus spatial stacking algorithm and novel training architecture that enabling in-depth automated analysis of 3D spheroid invasion behaviors within a microenvironment. Firstly, the MDA-MB-231 breast cancer spheroids were cultured in a microwell dish and then incorporated into a collagen type I matrix for the 3D spheroid invasion assay. Capturing spheroid invasion dynamics involved acquiring 15 z-plane images (spaced at 20 µm intervals) per timepoint for each spheroid using DIC microscopy at 15-minute intervals over a 28-hour period, employing a 20X objective lens. We refined the Focus Stacking framework into Multiple Slices Blurry Stacking (MSB-Stack) by pre-filtering blurred areas before stacking images. This process produced 6390 slices consolidated into 426 labeled stacked images, highlighting Invasive area, Spheroid core, and Invasive cells post-invasion. We then enhanced Mask R-CNN (Region based-Convolutional Neural Networks) with a weak-to-strong augmentation mechanism, leveraging well-segmented samples from MSB-Stack for training on both fully stacked and blurred images. Blurry Consistency Mask R-CNN (BCMask R-CNN) demonstrates overall improvement across feature extractors, outperforming Mask R-CNN. Using Mean average precision (mAP) as the main metric for instance segmentation and detection, our proposed model achieves 73.0% and 65.2% on mAP @[0.5, 0.95], showcasing robustness. Notably, our best results for mAP @0.5 reach 96.6% and 96.7% on both tasks, underscoring the effectiveness of consistency training in mitigating unclear boundary issues during preprocessing. In conclusion, our proposed method, MSB-Stack, effectively addresses the challenge of unclear boundaries in 3D breast cancer cell DIC microscopy datasets. The integration of weak-to-strong consistency training in our model, BCMask R-CNN, not only mitigates biases but also enhances performance across different backbones, setting the stage for future advancements in cell-tracking tasks and broader migratory analyses. Citation Format: Thi Kim Ngan Ngo, Thanh-Huy Nguyen, Fang-Yu Lin, Ting-Yuan Tu. A deep learning framework for automated segmentation and analysis of 3D breast cancer spheroid invasion dynamics [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2306.