Multi-site Validation Research Articles

Clinical validation of an AI-based pathology tool for scoring of metabolic dysfunction-associated steatohepatitis.

Metabolic dysfunction-associated steatohepatitis (MASH) is a major cause of liver-related morbidity and mortality, yet treatment options are limited. Manual scoring of liver biopsies, currently the gold standard for clinical trial enrollment and endpoint assessment, suffers from high reader variability. This study represents the most comprehensive multisite analytical and clinical validation of an artificial intelligence (AI)-based pathology system, AI-based measurement of metabolic dysfunction-associated steatohepatitis (AIM-MASH), to assist pathologists in MASH trial histology scoring. AIM-MASH demonstrated high repeatability and reproducibility compared to manual scoring. AIM-MASH-assisted reads by expert MASH pathologists were superior to unassisted reads in accurately assessing inflammation, ballooning, MAS ≥ 4 with ≥1 in each score category and MASH resolution, while maintaining non-inferiority in steatosis and fibrosis assessment. These findings suggest that AIM-MASH could mitigate reader variability, providing a more reliable assessment of therapeutics in MASH clinical trials.

Ethically Translating Advanced Neurotechnologies for Disorders of Consciousness: A Survey of Clinicians' Perspectives.

Although evaluation of disorders of consciousness (DoC) following brain injury has traditionally relied on bedside behavioral examination, advances in neurotechnology have elucidated novel approaches to detecting and predicting recovery of consciousness. Professional society guidelines now recommend that clinicians integrate these neurotechnologies into clinical practice as part of multimodal evaluations for some patients with DoC but have not crafted concrete protocols for this translation. Little is known about the experiences and ethical perspectives held by key stakeholder groups around the clinical implementation of advanced neurotechnologies to detect and predict recovery of consciousness. Recognizing this knowledge gap, the Data-Driven Neuroethics for Consciousness Detection (DECODE)survey examined clinicians' perspectives on advanced neurotechnologies for DoC care, including access to and rates of adoption, perceived utility, facilitators and barriers to adoption in clinical settings, ethical considerations surrounding clinical implementation, and challenges encountered in ensuring care for patients with acute and prolonged DoC. Mixed-methods analysis including qualitative analysis, grounded theory methodology, and ethical analysis was employed to assess responses and key themes. Ninety-two clinicians consented to the survey. More than 70% believed that standard bedside behavioral examination is insufficient, and nearly 60% viewed advanced neurotechnologies as integral in optimal DoC evaluation. Training gaps and limited institutional infrastructure were identified as salient barriers to clinical implementation. Thematic analysis revealed concerns about the interpretation of results, impact on surrogates, and validity of test results. Ethical themes of prognostic uncertainty, nihilism, and access also permeated multiple domains. Considerations surrounding access, knowledge base, results interpretation, and communication with surrogates are cross-cutting ethical threads shaping the clinical translation of advanced neurotechnologies for DoC. These components represent opportunities for implementation science work focused on democratizing access to neurotechnologies, educating clinicians on the use of novel techniques and interpretation of results, conducting multisite validation studies, and standardizing approaches to communicating test results.

Brain white matter pathways of resilience to chronic back pain: a multisite validation.

Chronic back pain (CBP) is a global health concern with significant societal and economic burden. While various predictors of back pain chronicity have been proposed, including demographic and psychosocial factors, neuroimaging studies have pointed to brain characteristics as predictors of CBP. However, large-scale, multisite validation of these predictors is currently lacking. In two independent longitudinal studies, we examined white matter diffusion imaging data and pain characteristics in patients with subacute back pain (SBP) over six- and 12-month periods. Diffusion data from individuals with CBP and healthy controls (HC) were analyzed for comparison. Whole-brain tract-based spatial statistics analyses revealed that a cluster in the right superior longitudinal fasciculus (SLF) tract had larger fractional anisotropy (FA) values in patients who recovered (SBPr) compared to those with persistent pain (SBPp), and predicted changes in pain severity. The SLF FA values accurately classified patients at baseline and follow-up in a third publicly available dataset (Area under the Receiver Operating Curve ∼ 0.70). Notably, patients who recovered had FA values larger than those of HC suggesting a potential role of SLF integrity in resilience to CBP. Structural connectivity-based models also classified SBPp and SBPr patients from the three data sets (validation accuracy 67%). Our results validate the right SLF as a robust predictor of CBP development, with potential for clinical translation. Cognitive and behavioral processes dependent on the right SLF, such as proprioception and visuospatial attention, should be analyzed in subacute stages as they could prove important for back pain chronicity.

The diagnostic performance of the Visitect Advanced Disease point-of-care CD4 platform: a pragmatic mixed-methods multisite validation, costing, and qualitative analysis.

The Visitect CD4 Advanced Disease test (AccuBio, Alva, United Kingdom) is a rapid, semi-quantitative assay that estimates CD4 results above or below 200 cells/μL. We evaluated the performance of the Visitect CD4 assay in semi-urban laboratories in Uganda. We performed a pragmatic laboratory validation of the Visitect CD4 platform in four routine HIV clinics in Uganda, nested within a cluster randomized trial evaluating an enhanced package of screening and treatment for persons with advanced HIV disease (NCT05085171). As part of the clinical trial, samples processed on the Visitect CD4 platform were confirmed using another CD4 testing method. We compared the diagnostic performance of the Visitect CD4 platform against the confirmatory method by evaluating the sensitivity, specificity, positive and negative predictive values. Of 1495 venous blood samples that were processed both by the Visitect CD4 test and another confirmatory CD4 platform at clinics in Kampala, Uganda, specificity was 81% (95% CI, 79%-84%) and the positive predictive value was 69% (95% CI, 66%-73%). There were no samples for which the Visitect test was >200 cells/μL and the confirmatory test was ≤200 cells/μL, resulting in a sensitivity of 100%. Among Visitect CD4 tests that were read as <200 cells/μL with confirmatory results >200 cells/μL, the median confirmatory CD4 result was 397 (IQR, 281-590) cells/μL. Specificity varied by clinic ranging from 63% to 99%. Given variable specificity of the Visitect CD4 Advanced Disease platform, successful implementation will require consideration of clinic context and laboratory staffing.

Validation of an AI-based solution for breast cancer risk stratification using routine digital histopathology images

BackgroundStratipath Breast is a CE-IVD marked artificial intelligence-based solution for prognostic risk stratification of breast cancer patients into high- and low-risk groups, using haematoxylin and eosin (H&E)-stained histopathology whole slide images (WSIs). In this validation study, we assessed the prognostic performance of Stratipath Breast in two independent breast cancer cohorts.MethodsThis retrospective multi-site validation study included 2719 patients with primary breast cancer from two Swedish hospitals. The Stratipath Breast tool was applied to stratify patients based on digitised WSIs of the diagnostic H&E-stained tissue sections from surgically resected tumours. The prognostic performance was evaluated using time-to-event analysis by multivariable Cox Proportional Hazards analysis with progression-free survival (PFS) as the primary endpoint.ResultsIn the clinically relevant oestrogen receptor (ER)-positive/human epidermal growth factor receptor 2 (HER2)-negative patient subgroup, the estimated hazard ratio (HR) associated with PFS between low- and high-risk groups was 2.76 (95% CI: 1.63–4.66, p-value < 0.001) after adjusting for established risk factors. In the ER+/HER2- Nottingham histological grade (NHG) 2 subgroup, the HR was 2.20 (95% CI: 1.22–3.98, p-value = 0.009) between low- and high-risk groups.ConclusionThe results indicate an independent prognostic value of Stratipath Breast among all breast cancer patients, as well as in the clinically relevant ER+/HER2- subgroup and the NHG2/ER+/HER2- subgroup. Improved risk stratification of intermediate-risk ER+/HER2- breast cancers provides information relevant for treatment decisions of adjuvant chemotherapy and has the potential to reduce both under- and overtreatment. Image-based risk stratification provides the added benefit of short lead times and substantially lower cost compared to molecular diagnostics and therefore has the potential to reach broader patient groups.

Utilizing ChatGPT for Curriculum Learning in Developing a Clinical Grade Pneumothorax Detection Model: A Multisite Validation Study.

Background: Pneumothorax detection is often challenging, particularly when radiographic features are subtle. This study introduces a deep learning model that integrates curriculum learning and ChatGPT to enhance the detection of pneumothorax in chest X-rays. Methods: The model training began with large, easily detectable pneumothoraces, gradually incorporating smaller, more complex cases to prevent performance plateauing. The training dataset comprised 6445 anonymized radiographs, validated across multiple sites, and further tested for generalizability in diverse clinical subgroups. Performance metrics were analyzed using descriptive statistics. Results: The model achieved a sensitivity of 0.97 and a specificity of 0.97, with an area under the curve (AUC) of 0.98, demonstrating a performance comparable to that of many FDA-approved devices. Conclusions: This study suggests that a structured approach to training deep learning models, through curriculum learning and enhanced data extraction via natural language processing, can facilitate and improve the training of AI models for pneumothorax detection.

Cerebrovascular reactivity MRI as a biomarker for cerebral small vessel disease-related cognitive decline: Multi-site validation in the MarkVCID Consortium.

Vascular contributions to cognitive impairment and dementia (VCID) represent a major factor in cognitive decline in older adults. The present study examined the relationship between cerebrovascular reactivity (CVR) measured by magnetic resonance imaging (MRI) and cognitive function in a multi-site study, using a predefined hypothesis. We conducted the study in a total of three analysis sites and 263 subjects. Each site performed an identical CVR MRI procedure using 5% carbon dioxide inhalation. A global cognitive measure of Montreal Cognitive Assessment (MoCA) and an executive function measure of item response theory (IRT) score were used as outcomes. CVR and MoCA were positively associated, and this relationship was reproduced at all analysis sites. CVR was found to be positively associated with executive function. The predefined hypothesis on the association between CVR and a global cognitive score was validated in three independent analysis sites, providing support for CVR as a biomarker in VCID. This study measured a novel functional index of small arteries referred to as cerebrovascular reactivity (CVR). CVR was positively associated with global cognition in older adults. This finding was observed in three independent cohorts at three sites. Our statistical analysis plan was predefined before beginning data collection.

Multisite Evaluation and Validation of Optical Genome Mapping for Prenatal Genetic Testing

Prenatal diagnostic testing of amniotic fluid, chorionic villi, or more rarely, fetal cord blood is recommended following a positive or unreportable noninvasive cell-free fetal DNA test, abnormal maternal biochemical serum screen, abnormal ultrasound, or increased genetic risk for a cytogenomic abnormality based on family history. Although chromosomal microarray is recommended as the first-tier prenatal diagnostic test, in practice, multiple assays are often assessed in concert to achieve a final diagnostic result. The use of multiple methodologies is costly, time consuming, and labor intensive. Optical genome mapping (OGM) is an emerging technique with application for prenatal diagnosis because of its ability to detect and resolve, in a single assay, all classes of pathogenic cytogenomic aberrations. In an effort to characterize the potential of OGM as a novel alternative to traditional standard of care (SOC) testing of prenatal samples, OGM was performed on a total of 200 samples representing 123 unique cases, which were previously tested with SOC methods (92/123=74.7% cases tested with at least two SOCs). OGM demonstrated an overall accuracy of 99.6% when compared with SOC methods, a positive predictive value of 100%, and 100% reproducibility between sites, operators, and instruments. The standardized workflow, cost-effectiveness, and high-resolution cytogenomic analysis demonstrate the potential of OGM to serve as a first-tier test for prenatal diagnosis.

CheckpointPx: A predictive radiology AI model of immune checkpoint inhibitor (ICI) benefit in non-small cell lung cancer (NSCLC).

8632 Background: Immune checkpoint inhibitors (ICIs) have dramatically transformed the field of non-small cell lung cancer (NSCLC) treatment. Despite the widespread use of PD-L1 as a biomarker in NSCLC, it has significant limitations as a reliable predictive biomarker for ICI response. Addressing this limitation, we have developed CheckpointPx, a non-invasive radiology AI tool to assist in ICI treatment selection for patients using only baseline CT scans. Here we demonstrate that CheckpointPx can predict improved outcomes specific to treatment with ICIs, presenting a new tool to address the shortage of reliable predictive ICI biomarkers in NSCLC and ultimately improve outcomes for patients undergoing immunotherapeutic interventions. Methods: CheckpointPx (v1.11) was trained on pre-treatment CT scans to predict response to ICIs from training data (D1: n=252 ICI recipients) from three institutions (A-C). In this study, we evaluated its performance in predicting response in two validation cohorts: a Treatment Dataset of patients receiving immunotherapy (D2: n=224, institutions B-D) and a Control Dataset of patients receiving platinum-based chemotherapy alone (D3: n=76, institution B). The response was defined as disease control per RECIST v1.1. Experienced radiologists and physicians delineated target lesions, and a deep learning model segmented pulmonary vessels. From these segmented regions, radiomic features were extracted using the Picture Health Px platform and were used to generate a radiomics benefit score and corresponding benefit groups using D1. The ability of benefit groups to stratify patients by progression-free survival (PFS) was compared across D2 and D3 to evaluate its utility in identifying patients who would benefit from ICI over chemotherapy. Results: D2 and D3 contained a mix of treatment lines (1st-4th) and predominantly late-stage tumors (Stage 3/4, &gt;85%). CheckpointPx included 16 features, such as measurements of intra-tumoral heterogeneity and vessel twistedness and branching patterns. Within the treatment dataset (D2), CheckpointPx was found to significantly stratify ICI patients by progression-free survival (PFS) with HR=0.68 (95% CI: 0.52 - 0.93, p=0.019). When applied to the control dataset, D3, benefit groups failed to stratify patients treated with chemotherapy by outcome (HR=0.91 [95% CI: 0.51-1.61], p=0.740), indicating that the signature was specific to ICI response. Conclusions: CheckpointPx demonstrated the ability to identify NSCLC patients who would benefit from ICI over chemo. The model’s association with PFS among ICI recipients, but not patients receiving chemotherapy alone, suggests that the signature is predictive of immunotherapy-related outcomes rather than generally prognostic. Additional independent, multi-site and prospective validation is warranted.

Association of artificial intelligence–derived collagen disorder architecture (CoDA) features with survival outcome and objective response to immune checkpoint inhibitors in patients with head and neck squamous cell carcinoma.

6061 Background: In several cancers, including head and neck squamous cell carcinoma (HNSCC), the immunosuppressive components of the tumor microenvironment (TME) can impact the effectiveness of immune checkpoint inhibitors (ICI). One significant component of the TME is the extracellular matrix, which is rich in collagen fibers. In this work we used machine learning and image analysis approaches on whole slide images (WSIs) to characterize collagen disorder architecture (CoDA) features and evaluated their association with outcomes in HNSCC patients receiving ICI. Methods: WSIs of HNSCC patients treated with ICI were obtained from University Hospitals (S1, n=43) and Emory University (S2, n=31). Tiles from the tumor annotated regions of the WSIs were extracted, and a derivative-of-Gaussian model was used to identify collagen fibers in the stroma of these tiles. Various CoDA features were then calculated as follows: (1) collagen fiber fragmentation measure, (2) collagen fiber bundling percentage, (3) collagen fiber rigidity measure, (4) collagen fiber anisotropy index and (5) collagen fiber density index. CoDA features of S1and S2 were combined and split into 50:50 for training and validation. For survival analysis using overall survival (OS) as endpoint, the median risk score in the training set was applied for risk stratification in the validation set by means of a Least Absolute Shrinkage and Selection Operator (LASSO) and Cox regression model. For predictive analysis, CoDA features from patients with objective response (OR) to ICI were identified (S1, non-responder=23, responder=20), (S2, non-responder=23, responder=8). The top features were then selected using the LASSO and combined with a Generalized Linear Model classifier. A 5-fold cross-validation assessed Area Under the Receiver Operating Characteristics Curve (AUC) for predicting OR, with average AUC as the final performance metric in the validation set. Results: For survival analysis, high risk patients in the validation set had worse survival than low risk patients (HR=2.7 (95% CI=1.1-6.6, p=0.02)). For predicting OR, the selected top CoDA features were collagen fiber fragmentation measure, collagen fiber bundling percentage, collagen fiber rigidity measure and collagen fiber density index and the average AUC was 0.64±0.16. More fragmentation of the collagen fibers along with dense thick bundles and straightened fibers were observed in the WSIs of non-responder patients to ICI. Conclusions: High risk CoDA features correlated with worse survival in patients with HNSCC receiving ICI. Also, we established a correlation of specific CoDA features with OR to ICI. The prognostic and predictive value of CoDA deserves additional exploration with confirmatory data from larger, independent multi-site validation.

A multisite validation of a two hours antibiotic susceptibility flow cytometry assay directly from positive blood cultures

BackgroundRapid antimicrobial susceptibility testing (AST) is urgently needed to provide safer treatment to counteract antimicrobial resistance. This is critical in septic patients, because resistance increases empiric therapy uncertainty and the risk of a poor outcome. We validate a novel 2h flow cytometry AST assay directly from positive blood cultures (PBC) by using a room temperature stable FASTgramneg and FASTgrampos kits (FASTinov® Porto, Portugal) in three sites: FASTinov (site-1), Hospital Ramon y Cajal, Madrid, Spain (site-2) and Centro Hospitalar S. João, Porto, Portugal (site-3). A total of 670 PBC were included: 333 spiked (site-1) and 337 clinical PBC (151 site-2 and 186 site-3): 367 gram-negative and 303 gram-positive. Manufacturer instructions were followed for sample preparation, panel inoculation, incubation (1h/37ºC) and flow cytometry analysis using CytoFlex (Site-1 and -2) or DxFlex (site-3) both instruments from Beckman-Coulter, USA.ResultsA proprietary software (bioFAST) was used to immediately generate a susceptibility report in less than 2 h. In parallel, samples were processed according to reference AST methods (disk diffusion and/or microdilution) and interpreted with EUCAST and CLSI criteria. Additionally, ten samples were spiked in all sites for inter-laboratory reproducibility. Sensitivity and specificity were >95% for all antimicrobials. Reproducibility was 96.8%/95.0% for FASTgramneg and 95.1%/95.1% for FASTgrampos regarding EUCAST/CLSI criteria, respectively.ConclusionFASTinov® kits consistently provide ultra-rapid AST in 2h with high accuracy and reproducibility on both Gram-negative and Gram-positive bacteria. This technology creates a new paradigm in bacterial infection management and holds the potential to significantly impact septic patient outcomes and antimicrobial stewardship.

Abstract PO3-07-05: Multi-site validation of a deep learning solution for ER/PR profiling of breast cancer from H&amp;E-stained pathology slides

Abstract Background: Molecular profiling of estrogen and progesterone receptors (ER/PR/Her2) is performed for all malignant breast cancers to inform the choice of targeted therapy. Though existing scoring systems are widely used and well-validated, they can involve costly preparation and variable interpretation. Additionally, discordances between histology and expected biomarker findings can prompt repeat testing to address biological, interpretative, or technical reasons for unexpected results. We evaluate PANProfiler Breast(PPB), a UKCA/CE- IVDD marked deep learning (DL)-based image analysis software, on multiple sites to determine if the majority of ER/PR assays can be replaced, relying only on routinely-used H&amp;E-stained whole slide images. Methods: PPB was trained and validated on 5126/4619 WSIs from 5 sites to identify the ER/PR status defined by IHC assays graded in alignment with ASCO/RCPATH guidelines from five different sites in the UK were used for training and validation. The performance is evaluated separately for each site with 3-fold cross-validation, mimicking real-world distribution. Results: For ER, with a class ratio (CR) of approximately 4:1, we measure a sensitivity, specificity, and accuracy of 95.5%(±1.5%), 47.5%(±15.1%) and 88.3%(±0.6%) averaged over all sites, reaching up to 97.40%, 67.20%, and 89.30% respectively. For PR (CR approx. 3:1), the averaged sensitivity, specificity, and accuracy are 92.2%(±8.4%), 53.10%(±20.8%), and 86.6%(±2.4%), reaching up to 99.1%, 81.7%, and 88.8%, respectively. The software's performance is comparable to current SoC antibody performance in common ER/PR CDx Assays from Dako, Leica, and Roche, which have sensitivities of 98.5%(±1.3%) and specificities of 38.6%(±6.3%) for ER and sensitivities of 96.9%(±0.6%) and specificities of 23.4%(±1.4%) for PR. Performance was robust to specimen and scanner types, with accuracies of 87.6% (ER, only biopsies), 88.2% (ER, only resections), 88.5% (ER mixed types), 83.9% (PR, only resections) and 87.3% (mixed types). Accuracy across scanners varied by a standard deviation of 0.3%/1.0% for ER/PR respectively. Conclusions: We demonstrate the robustness of a DL-based ER/PR profiling method in breast cancer using only H&amp;E-stained WSIs. This multi-site validation study is the first-of-its-kind for such an approach using real-world clinical data. Our solution could facilitate fast, accurate, and systemic screening of patients for targeted treatments if integrated into routine pathological workflows. Citation Format: Salim Arslan, Adrian Bazaga, Gareth Bryson, Oscar Carlos, Andre Geraldes, David Harrison, Alastair Ironside, Jakob Kather, Ali Khurram, David Leff, Debapriya Mehrotra, Foivos Ntelemis, John Nyonyintono, Julian Schmidt, Shikha Singhal, In Hwa Um, Steffen Wolf, Pahini Pandya. Multi-site validation of a deep learning solution for ER/PR profiling of breast cancer from H&amp;E-stained pathology slides [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO3-07-05.

Diffusion deep learning for brain age prediction and longitudinal tracking in children through adulthood

Abstract Deep learning (DL)-based prediction of biological age in the developing human from a brain magnetic resonance imaging (MRI) (“brain age”) may have important diagnostic and therapeutic applications as a non-invasive biomarker of brain health, aging, and neurocognition. While previous deep learning tools for predicting brain age have shown promising capabilities using single-institution, cross-sectional datasets, our work aims to advance the field by leveraging multi-site, longitudinal data with externally validated and independently implementable code to facilitate clinical translation and utility. This builds on prior foundational efforts in brain age modeling to enable broader generalization and individual’s longitudinal brain development. Here, we leveraged 32,851 T1-weighted MRI scans from healthy children and adolescents aged 3 to 30 from 16 multisite datasets to develop and evaluate several DL brain age frameworks, including a novel regression diffusion DL network (AgeDiffuse). In a multisite external validation (5 datasets), we found that AgeDiffuse outperformed conventional DL frameworks, with a mean absolute error (MAE) of 2.78 years (interquartile range [IQR]: [1.2-3.9]). In a second, separate external validation (3 datasets), AgeDiffuse yielded an MAE of 1.97 years (IQR: [0.8-2.8]). We found that AgeDiffuse brain age predictions reflected age-related brain structure volume changes better than biological age (R2 = 0.48 vs. R2 = 0.37). Finally, we found that longitudinal predicted brain age tracked closely with chronological age at the individual level. To enable independent validation and application, we made AgeDiffuse publicly available and usable for the research community.

Abstract 7669: Use of computational pathology to predict biochemical recurrence in prostate cancer (Pca) (Rp) patients following radical prostatectomy: Evaluation in low decipher risk category

Abstract Background: Current tools for predicting prostate cancer biochemical recurrence (BCR) after RP often depend on parameters determined by pathologists, such as tumor grade, which is known to vary between reviewers. Genomic risk classifiers provide useful information but require substantial tissue, are costly, and are not commonly accessible in many medical centers. The study presents an artificial intelligence (AI) model for automated risk assessment of biochemical recurrence (BCR) by analyzing nuclear morphologic patterns from Pca archival H&amp;E slides. Nuclear morphometric features are essential for disease diagnosis and gaining insights into cellular functions. Specifically, we sought to evaluate whether the computational pathology approach could provide additional granular risk stratification within the low-risk group identified by Decipher. Methods: The study employed two cohorts from the University of Pennsylvania D1 and D2. D1 comprised 180 patients, while D2 comprised 175 patients, including 63 individuals from the Decipher low-risk group. None of the patients in either cohort had received any pre-operative treatments. Within the Decipher cohort, a tumor segmentation model was created using the U-net deep learning network. The HoVer-Net model was used to both segment and classify nuclei. The feature extracted included a combination of nuclear texture and quantitative measurements of the spatial arrangement of nuclei. The eleven most significant features, selected via the least absolute shrinkage and selection operator (LASSO), were utilized to build a Cox regression model for predicting the risk of BCR in D1. In D2, patients were classified as either low-risk or high-risk for BCR based on the median risk score derived from D1. Results: Patients from D1 identified as high risk by the AI model experienced shorter survival, with a median BCR time of 27 months, in contrast to 55 months for those classified as low risk. The AI model demonstrated significant prognostic value for BCR in both D1 (HR = 6.82, 95% confidence interval [CI]: 3.54-13.12, p &amp;lt; 0.0019), and D2 (HR = 2.72, 95% CI: 1.53-4.82, p &amp;lt; 0.0032). Furthermore, the model effectively stratified the Decipher low-risk group in D2 into distinct low and high-risk categories (HR = 4.52, 95% confidence interval [CI]: 1.70-12.05, p &amp;lt; 0.0085). The most prognostic features identified included five related to nuclei shape diversity (Minor Axis Length, Eccentricity, Equivalent Diameter, Solidity, and Circularity) in conjunction with nine nuclear texture features. Conclusions: The AI model's findings suggest that a digital image-based prognostic classifier for prostate cancer could serve as an alternative or complementary method to molecular-based companion risk tests. Additional, independent multi-site and prospective validation of these findings are warranted. Citation Format: Kamal Hammouda, Tilak Pathak, Tuomas Mirtti, Priti Lal, Shilpa Gupta, Anant Madabhushi. Use of computational pathology to predict biochemical recurrence in prostate cancer (Pca) (Rp) patients following radical prostatectomy: Evaluation in low decipher risk category [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7669.

Abstract 3516: AI-derived features of 3D glandular networks are associated with likelihood of biochemical recurrence post radical prostatectomy

Abstract Introduction: Prostate cancer is the most common cancer in American men and is mainly diagnosed from two-dimensional histology sections. Recently, three-dimensional (3D) prostate histopathology has emerged as a valuable strategy to enhance the understanding of the disease by increasing microscopic sampling of specimens. This strategy enhances the microscopic examination of specimens by facilitating the analysis of the volumetric shape of cells and glands, as well as the pathway structures through entire biopsies and other morphological features. These aspects hold the potential to be linked with biochemical recurrence (BCR) outcomes. This study highlights the potential of features derived from the 3D structure of glands in samples of patients with prostate cancer for identifying patients who are at higher risk of BCR. Methods: Radical prostatectomy specimens were collected from 50 patients along with 5-year postoperative BCR follow-up from the University of Washington. A total of 118 ex vivo whole-biopsy 3D pathology images were obtained from RP by an open-top light-sheet microscopy platform. Images were converted into a synthetic CK8 immunofluorescence dataset using an image-sequence translation model. From these 3D images, a glandular structure mask was generated by a computer-vision algorithm. The 3D structure of prostate glands was modeled through a Tree-structure Extraction Algorithm for Accurate and Robust Skeletons (TEASER), which provides a simplification of the length of the gland, its shape, and its pathway. 91 features were then extracted from the glandular skeleton architecture. A 3-fold cross-validation approach was used in which patients were divided into training (D1) and test (D2) sets of 33 and 17 patients, respectively. Fisher’s score, used to select features based on their correlation to the outcome, was applied to D1 to identify the top 5 features most correlated with BCR. We then utilized these features to train a K-Nearest Neighbors classifier on D1, aimed at predicting whether a patient would experience BCR. The model’s performance was evaluated on D2 and averaged across all folds. Results: The classifier was able to accurately differentiate between BCR+ and BCR- patients, achieving an average ROCAUC of 0.73 (±0.04) in D2. Significant differences in 3D skeletal features were observed between BCR+ and BCR- groups (p&amp;lt;0.05). Among the five selected features, three were related to the length and diameter of the gland, with the longest glands observed in BCR- patients. The remaining two features pertained to gland curvature and tortuosity, revealing more twisted glands in BCR+ patients. Conclusion: Features derived from a skeleton model of the 3D glandular structure of biopsies showed promise in identifying patients with prostate cancer who are at a higher risk of BCR. Additional, multi-site independent validation of these findings is warranted. Citation Format: Jennifer Salguero, Hisham Abdeltawab, Kamal Hammouda, German Corredor, Sarah Hawley, Pushkar Mutha, Rohan Dhamdhere, Sebastian Medina, Tilak Pathak, Kevin Bishop, Rob Serafin, Tuomas Mirtti, Priti Lal, Lawrence True, Jonathan Liu, Anant Madabhushi. AI-derived features of 3D glandular networks are associated with likelihood of biochemical recurrence post radical prostatectomy [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3516.

Abstract 1025: An AI based CT scan classifier for predicting extranodal extension and nodal metastasis of HPV-associated oropharyngeal carcinoma

Abstract Background: Pathologic extranodal extension (ENE) and nodal metastasis (NM) are of prognostic significance in human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma (OPSCC). The pretreatment identifications of these two factors through computed tomography (CT) imaging are of clinical importance since they could guide treatment de-escalation protocols and select candidates suitable for surgery. In an effort to enhance pretreatment decision making, we developed and validated an AI model (specifically a multi-layer perceptron (MLP)-based deep learning model) to predict radiologic NM and pathologic ENE in HPV-associated OPSCC patients. Methods: Radiographic CT scans for 290 OPSCC patients treated with definitive chemoradiation were acquired from The Cancer Imaging Archive (TCIA) and divided into training (D1) and validation (D2) cohorts for NM prediction. Radiologic NM for D1 and D2 were reviewed by two radiologists. Only nodes with concordant NM judgments from both radiologists were included for analysis. An additional 46 OPSCC patients treated with surgery at Emory Winship Cancer Institute were collected to form the test cohorts (D3) for ENE prediction. The largest metastatic or normal nodes were annotated for D1 and D2 while all visible nodes with pathology confirmation of ENE from D3 were annotated. We present a novel deep-learning model utilizing U-shaped multi-layer perceptron mixer (MLP-Mixer) to capture both global and local feature representations by spatially aggregating image patch information through a procedure called token-mixing. The model was trained for 20 epochs with learning rate of0.0001 and batch size of 64. The diagnostic performance of the AI model was evaluated on D2 and D3 using accuracy and area under the ROC curve (AUROC). Results: There were 121 NM negative and 130 NM positive nodes in D1, 14 NM negative and 25 NM positive nodes in D2 and 46 ENE negative and 20 ENE positive nodes in D3. On D2 for NM prediction, the AI model achieved overall accuracy of 0.77 and AUROC of 0.78. On D3 for ENE prediction, the AI model achieved overall accuracy of 0.71, AUROC of 0.69, positive predictive value of 0.52 and negative predictive value of 0.86. Conclusions: The new AI–based deep learning model was able to identify HPV-associated OPSCC NM and ENE with satisfactory diagnostic performance and shows promise to assist clinicians in treatment decision-making. Additional, multi-site prospective validation of the approach is warranted. Citation Format: Bolin Song, Kailin Yang, Shaoyan Pan, Liping Li, Qiyun Liu, Jonathan Lee, Sarah Stock, Nabil F. Saba, Mehmet Asim Bilen, Mihir R. Patel, Anant Madabhushi. An AI based CT scan classifier for predicting extranodal extension and nodal metastasis of HPV-associated oropharyngeal carcinoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1025.

Diagnosing schizophrenia using deep learning: Novel interpretation approaches and multi-site validation

Schizophrenia is a profound and enduring mental disorder that imposes significant negative impacts on individuals, their families, and society at large. The development of more accurate and objective diagnostic tools for schizophrenia can be expedited through the employment of deep learning (DL), that excels at deciphering complex hierarchical non-linear patterns. However, the limited interpretability of deep learning has eroded confidence in the model and restricted its clinical utility. At the same time, if the data source is only derived from a single center, the model's generalizability is difficult to test. To enhance the model's reliability and applicability, leave-one-center-out validation with a large and diverse sample from multiple centers is crucial. In this study, we utilized Nine different global centers to train and test the 3D Resnet model's generalizability, resulting in an 82% classification performance (area under the curve) on all datasets sourced from different countries, employing a leave-one-center-out-validation approach. Per our approximation of the feature significance of each region on the atlas, we identified marked differences in the thalamus, pallidum, and inferior frontal gyrus between individuals with schizophrenia and healthy controls, lending credence to prior research findings. At the same time, in order to translate the model's output into clinically applicable insights, the SHapley Additive exPlanations (SHAP) permutation explainer method with an anatomical atlas have been refined, thereby offering precise neuroanatomical and functional interpretations of different brain regions.

Multisite development and validation of machine learning models to predict severe outcomes and guide decision-making for emergency department patients with influenza.

Millions of Americans are infected by influenza annually. A minority seek care in the emergency department (ED) and, of those, only a limited number experience severe disease or death. ED clinicians must distinguish those at risk for deterioration from those who can be safely discharged. We developed random forest machine learning (ML) models to estimate needs for critical care within 24h and inpatient care within 72h in ED patients with influenza. Predictor data were limited to those recorded prior to ED disposition decision: demographics, ED complaint, medical problems, vital signs, supplemental oxygen use, and laboratory results. Our study population was comprised of adults diagnosed with influenza at one of five EDs in our university health system between January 1, 2017 and May 18, 2022; visits were divided into two cohorts to facilitate model development and validation. Prediction performance was assessed by the area under the receiver operating characteristic curve (AUC) and the Brier score. Among8032 patientswith laboratory-confirmed influenza,incidence of critical care needs was 6.3% and incidence of inpatient care needswas 19.6%.Themost common reasons for ED visitwere symptoms of respiratory tract infection,fever,and shortness of breath. Model AUCswere0.89 (95% CI 0.86-0.93)for prediction of criticalcare and0.90(95% CI0.88-0.93)for inpatient care needs;Brier scores were0.026 and0.042, respectively.Importantpredictorsincludedshortness of breath,increasingrespiratory rate,and ahigh number of comorbid diseases. ML methods can be used to accurately predict clinical deterioration in ED patients with influenza and have potential to support ED disposition decision-making.

Protocol improvement and multisite validation of a digital soft agar colony formation assay for tumorigenic transformed cells intermingled in cell therapy products

Background aimsThe administration of human cell-processed therapeutic products (hCTPs) is associated with a risk of tumorigenesis due to the transformed cellular contaminants. To mitigate this risk, these impurities should be detected using sensitive and validated assays. The digital soft agar colony formation (D-SAC) assay is an ultrasensitive in vitro test for detecting tumorigenic transformed cells in hCTPs. Methods: In this study, we first evaluated the colony formation efficiency (CFE) precision of tumorigenic reference cells in positive control samples according to a previously reported D-SAC assay protocol (Protocol I) from multiple laboratories. However, the CFE varied widely among laboratories. Thus, we improved and optimized the test protocol as Protocol II to reduce variability in the CFE of tumorigenic reference cells. Subsequently, the improved protocol was validated at multiple sites. Human mesenchymal stromal cells (hMSCs) were used as model cells, and positive control samples were prepared by spiking them with HeLa cells. Results: Based on the previously reported protocol, the CFE was estimated using an ultra-low concentration (0.0001%) of positive control samples in multiple plates. Next, we improved the protocol to reduce the CFE variability. Based on the CFE results, we estimated the sample size as the number of wells (Protocol II) and assessed the detectability of 0.0001% HeLa cells in hMSCs to validate the protocol at multiple sites. Using Protocol I yielded low CFEs (mean: 30%) and high variability between laboratories (reproducibility coefficient of variance [CV]: 72%). In contrast, Protocol II, which incorporated a relatively high concentration (0.002%) of HeLa cells in the positive control samples, resulted in higher CFE values (mean: 63%) and lower variability (reproducibility CV: 18%). Moreover, the sample sizes for testing were estimated as the number of wells per laboratory (314–570 wells) based on the laboratory-specific CFE (42–76%). Under these conditions, all laboratories achieved a detection limit of 0.0001% HeLa cells in hMSCs in a predetermined number of wells. Moreover, colony formation was not observed in the wells seeded with hMSCs alone. ConclusionsThe D-SAC assay is a highly sensitive and robust test for detecting malignant cells as impurities in hCTPs. In addition, optimal assay conditions were established to test tumorigenic impurities in hCTPs with high sensitivity and an arbitrary false negative rate.

Development and multi-site external validation of a generalizable risk prediction model for bipolar disorder

Bipolar disorder is a leading contributor to disability, premature mortality, and suicide. Early identification of risk for bipolar disorder using generalizable predictive models trained on diverse cohorts around the United States could improve targeted assessment of high risk individuals, reduce misdiagnosis, and improve the allocation of limited mental health resources. This observational case-control study intended to develop and validate generalizable predictive models of bipolar disorder as part of the multisite, multinational PsycheMERGE Network across diverse and large biobanks with linked electronic health records (EHRs) from three academic medical centers: in the Northeast (Massachusetts General Brigham), the Mid-Atlantic (Geisinger) and the Mid-South (Vanderbilt University Medical Center). Predictive models were developed and valid with multiple algorithms at each study site: random forests, gradient boosting machines, penalized regression, including stacked ensemble learning algorithms combining them. Predictors were limited to widely available EHR-based features agnostic to a common data model including demographics, diagnostic codes, and medications. The main study outcome was bipolar disorder diagnosis as defined by the International Cohort Collection for Bipolar Disorder, 2015. In total, the study included records for 3,529,569 patients including 12,533 cases (0.3%) of bipolar disorder. After internal and external validation, algorithms demonstrated optimal performance in their respective development sites. The stacked ensemble achieved the best combination of overall discrimination (AUC = 0.82–0.87) and calibration performance with positive predictive values above 5% in the highest risk quantiles at all three study sites. In conclusion, generalizable predictive models of risk for bipolar disorder can be feasibly developed across diverse sites to enable precision medicine. Comparison of a range of machine learning methods indicated that an ensemble approach provides the best performance overall but required local retraining. These models will be disseminated via the PsycheMERGE Network website.