Abstract 4933: A workflow for cloud-based AI development of multiplex IF image analysis using the OMERO Plus platform

Abstract

Abstract Background: The advancement of AI in digital pathology is essential for analyzing image data containing billions of cells. This is true for digital pathology workflows using bright-field images, but this is even more critical for multiplex immunofluorescence (mIF) whole-slides due to the increased number of fluorescent channels and bit-depth per pixel. These complexities necessitate improved data management, and efficient visualization for both the mIF image and AI-based analysis results. We present a workflow using OMERO Plus, AWS SageMaker, and Ultivue software for AI model development in mIF. Method: Images were uploaded to AWS S3 and imported to the data management system OMERO Plus, where they were organized into training and validation groups. Annotations: PathViewer was used to place rectangular fields of view (FOV) in locations selected by the pathologist where all positive cells for each target marker were manually annotated. Data Preparation: Using the OMERO Plus API, annotation coordinates, image tiles, and image metadata were extracted and formatted for model training. All training data is under a version control system using Gitlab and Data Version Control tools using AWS S3. Additionally, strict quality checks were performed to ensure data integrity. Training: For both semantic segmentation and cell object detection, the training data was used in a customized training workflow using AWS SageMaker, including modern deep learning architectures, data augmentation, training logging, evaluation, and deployment. All AI models were trained using Pytorch and deployed with the open model format ONNX. Results Visualization: AI segmentation results of cells and regions are visualized using PathViewer as overlays on the original images by converting them to the OME-NGFF label image format that uses Zarr multidimensional arrays. Results: We utilized 58 and 24 mIF images for training and validation, respectively, on 15 markers using ISP Ultivue technology with 1681 and 296 FOVs, encompassing over 60K cells. Using this data and methodology we trained 3 semantic segmentation models and over five object detection models suitable for all twenty-four OmniVUE markers. For instance, the CD3 positive cell detection model, trained with 390 FOVs, achieved highly accurate results: f1-score of 0.915 evaluated on 34 FOVs. Conclusions: The combination of OMERO Plus, PathViewer, AWS SageMaker, and internal tools can successfully address mIF Big Data data management complexities in a cloud environment for digital pathology. The architecture is highly scalable, cost effective, and reliable, enabling a very efficient AI workflow. Citation Format: Ruben Cardenes, Erin Diel, Douglas Wood, Je Lee, Angela Vasaturo, Martin Schulze, Lorenz Rognoni. A workflow for cloud-based AI development of multiplex IF image analysis using the OMERO Plus platform [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4933.