Revisiting ADMET prediction reliability under real-world challenges in the foundation model era.

1. Rationale: Lung squamous cell carcinoma (LUSC) is aggressive with poor prognosis, making survival analysis crucial for personalized treatment. Integrating diverse data modalities—from genetic to organ levels—is challenging yet may provide synergistic insights. Traditional methods struggle with heterogeneous data and missing modalities, leading to suboptimal predictions. To address these challenges, we propose an embeddings-based relational learning framework using Foundation Models (FMs) and Graph Neural Networks (GNNs) to integrate diverse data—including electronic health records (EHR), whole slide images (WSI), pathology reports, and molecular data—into a unified graph representation.2. Methods: We analyzed two LUSC cohorts: The Cancer Genome Atlas (TCGA) with 496 patients and Moffitt Cancer Center with 103 patients. Data included EHR variables (age, gender, etc.), WSI, pathology reports, and molecular data (gene expression, miRNA expression, DNA methylation, mutations, protein expression). Each modality underwent tailored preprocessing; for example, WSI patches were extracted and normalized, pathology reports were processed using natural language processing, and molecular data were denoised and dimension-reduced. Modality-specific FMs generated embeddings: GatorTron for EHR and pathology reports, UNI FM for WSI, and SeNMo FM for molecular data. Sample-level embeddings were aggregated into patient-level representations using pooling or multiple instance learning and fused into a unified graph where nodes represent patients and edges capture relationships based on similarity measures like Euclidean distance and k-nearest neighbors. We implemented GNNs tailored for survival analysis, trained using the Cox proportional hazards model. Performance was evaluated using the average concordance index (C-index) with 7-fold cross-validation, comparing our model against baseline models including Multilayer Perceptron (MLP), Transformers, and XGBoost.3. Results: Our multimodal GNN outperformed baseline models, achieving average C-indexes of 0.72 (TCGA) and 0.93 (Moffitt), compared to next best scores of 0.62 (XGBoost) and 0.91 (Transformers). The GNN effectively integrated heterogeneous modalities, capturing complex interactions missed by single-modality models. Combining all data modalities improved predictive performance, as observed in Moffitt's LUSC results. The embeddings-based approach was resilient to missing modalities, maintaining performance even when some data types were absent. Visualizations of the patient graph revealed clusters with similar survival outcomes, indicating meaningful relationships were captured.4. Conclusion: We present a novel embeddings-based relational learning framework using FMs and GNNs for LUSC survival analysis. Improved predictive performance over baseline models highlights the value of capturing relational structures and complex interactions. Converging individual data modalities provides comprehensive disease understanding, leading to more accurate predictions and potential discovery of novel biomarkers and therapeutic targets.

Introduction: Cancer research faces significant challenges in integrating heterogeneous data across varying spatial and temporal scales, limiting the ability to gain a comprehensive understanding of the disease. PARADIGM(Pan-Cancer Embeddings Representation using Advanced Multimodal Learning with Graph-based Modeling) addresses this challenge by providing a framework leveraging foundation models (FMs) and Graph Neural Networks (GNN). PARADIGM framework generates embeddings from multi-resolution datasets using modality-specific FMs, aggregates sample embeddings, fuses them into a unified graph representation, and uses GNNs for survival prediction. Methods: We applied PARADIGM for survival analysis of five squamous cell carcinomas; head and neck (527), lung (496), bladder (408), cervical (294), and esophageal (184) cancers, and five adenocarcinomas; colon (393), lung (549), prostate (490), rectum (157), and stomach (406) cancers. Additionally, we validated using lung squamous cell data from Moffitt Cancer Center (103 patients). Multimodal data for training and evaluation included EHR data (e.g., age, gender, and smoking status), whole slide images (WSIs), pathology reports, and molecular data (e.g., gene expression, miRNA, and protein expression). Model performance was assessed using concordance index (C-index) for survival prediction, with 7-fold cross-validation ensuring robustness. Results & Conclusions: PARADIGM demonstrated significant improvements in survival prediction compared to unimodal and multimodal approaches such as multilayer perceptron, Transformers, and XGBoost (Table 1). Our findings indicate that integrating diverse data modalities into a cohesive representation yields more accurate and insightful predictions. This approach highlights the potential of embeddings-based multimodal learning to foster improved outcome predictions and provide actionable insights for personalized treatment strategies. Citation Format: Asim Waqas, Aakash Tripathi, Mia Naeini, Paul A. Stewart, Matthew B. Schabath, Ghulam Rasool. PARADIGM: an embeddings-based multimodal learning framework with foundation models and graph neural networks [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 991.

Histopathological image classification using computational methods such as fine-tuned convolutional neural networks (CNNs) has gained significant attention in recent years. Graph neural networks (GNNs) have also emerged as strong alternatives, often employing CNNs or vision transformers (ViTs) as node feature extractors. However, as these models are usually pre-trained on small-scale natural image datasets, their performance in histopathology tasks can be limited. The introduction of foundation models trained on large-scale histopathological data now enables more effective feature extraction for GNNs. In this work, we integrate recently developed foundation models as feature extractors within a lightweight GNN and compare their performance with standard fine-tuned CNN and ViT models. Furthermore, we explore a prediction fusion approach that combines the outputs of the best-performing GNN and fine-tuned model to evaluate the benefits of complementary representations. Results demonstrate that GNNs utilizing foundation model features outperform those trained with CNN or ViT features and achieve performance comparable to standard fine-tuned CNN and ViT models. The highest overall performance is obtained with the proposed prediction fusion strategy. Evaluated on three publicly available datasets, the best fusion achieved F1-scores of 98.04%, 96.51%, and 98.28%, and balanced accuracies of 98.03%, 96.50%, and 97.50% on PanNuke, BACH, and BreakHis, respectively.

The rapid advancement of foundation models has transformed the landscape of machine learning by enabling scalable, general-purpose solutions across diverse domains such as natural language processing, computer vision, and multimodal reasoning. Simultaneously, Graph Neural Networks (GNNs) have become the de facto standard for learning over structured data due to their ability to model complex relational dependencies inherent in graphs. The convergence of these two paradigms—GNNs and foundation models—marks a significant milestone in the pursuit of universal representation learning over graph-structured data. This survey presents a comprehensive and in-depth exploration of Graph Neural Networks in the context of large-scale foundation models, highlighting key methodologies, architectural innovations, training strategies, applications, and open research challenges. We begin by reviewing the mathematical underpinnings of graph neural architectures, including message passing, graph convolution, and attention mechanisms. Building on this foundation, we explore how self-supervised pretraining techniques—such as masked node and edge prediction, contrastive learning, and graph autoencoding—have been adapted to equip graph models with the capacity for transfer learning and generalization. We further analyze architectural trends that scale GNNs to foundation model capacities, including Graph Transformers, scalable neighborhood sampling methods, and structural encoding schemes. A detailed overview of applications illustrates the versatility of graph foundation models in real-world scenarios, including drug discovery, knowledge graph reasoning, recommender systems, scientific simulations, and fraud detection. In addressing current limitations, we examine critical challenges such as computational scalability, data availability, heterogeneity, dynamic graph modeling, interpretability, and ethical considerations. We also identify key directions for future research, including the development of universal graph encoders, integration with other modalities, lifelong graph learning, and responsible AI deployment. By synthesizing recent advances and outlining a forward-looking roadmap, this survey aims to provide both a foundational reference and a strategic perspective for researchers, practitioners, and developers working at the intersection of graphs and large-scale machine learning. Ultimately, we argue that graph foundation models are poised to play a central role in the next generation of AI systems, enabling machines to reason over structured relational data with unprecedented depth, scale, and generality.

The invention of single-cell RNA sequencing (scRNA-seq) has enabled transcriptomic examination of cells on an individual basis, uncovering cell-to-cell phenotypic heterogeneity within isogenic cell populations and tissues. Inevitably, cell type annotation has emerged as a fundamental, albeit challenging task in scRNA-seq data analysis, which involves identifying and characterizing cells based on their unique molecular profiles. Recently, deep learning techniques with their data-driven priors have shown significant promise in this task. On the one hand, task-agnostic transformer networks pre-trained on large-scale biological databases can capture generalizable data representations to serve as foundation models despite their ineffectiveness in characterizing intricate relationships between biological entities such as cells or genes. Contrarily, task-specific graph neural networks (GNNs) can be trained on target datasets to sensitively characterize entity relationships, but they can suffer from relatively poor generalizability. Furthermore, existing GNNs focus exclusively on either homogeneous or heterogeneous relationships, limiting their ability to offer a complete picture of the diverse inner structure of cells. In this study, we propose a novel merged transformer-graph model, scGraPhT, that integrates a pre-trained transformer to extract rich representations of scRNA-seq data with a multi-layered GNN to capture cell-cell, cell-gene, and gene-gene relationships. Different from previous GNNs, scGraPhT examines both homogeneous and heterogeneous relationships through subgraph layers to offer a more comprehensive assessment. Since the graph construction in scGraPhT relies on representations from a pre-trained transformer model, our approach does not require costly training procedures. Moreover, scGraPhT can also be adapted to leverage any transformer-based single-cell annotation method, such as scGPT or scBERT, that produces suitable embedding representations as input for GNNs. Demonstrations on three benchmark scRNA-seq datasets indicate that scGraPhT outperforms state-of-the-art annotation methods without compromising efficiency. We offer insights into performance improvements by employing Grad-CAM, a visual explainability method that elucidates the complementary nature of the GNN- and transformer-based components of scGraPhT to improve its predictive performance.

The candidate–job matching (CJM) problem, central to high-skill recruitment in domains like technology, management, and finance, has seen rapid progress through machine learning (ML) since 2021. Modern systems move beyond simple keyword matching, leveraging advanced natural language processing (NLP), graph representations, and hybrid recommender methods. Transformer-based models (e.g. BERT and derivatives) now embed resumes and job descriptions into semantic spaces, enabling nuanced similarity comparisons. Graph neural networks (GNNs) capture rich relationships among candidates, skills, and jobs, often outperforming traditional neural models in screening tasks. Classical ML approaches (e.g. support vector machines, tree ensembles) remain useful for structured feature matching but are complemented by deep models for unstructured text. Recommender-system techniques – including collaborative filtering, content-based filtering, and hybrid designs – incorporate contextual signals (experience, industries, user behaviors) to improve personalization. Reviewed benchmarks report that fine-tuned transformers and GNNs can significantly boost ranking accuracy (e.g. ~15% NDCG improvements [1]) and screening sensitivity (e.g. GNN balanced accuracy 65.4% vs 55.0% for a plain MLP [2]). These gains come with challenges: neural approaches often act as black boxes, raising interpretability concerns, and large models incur high computational costs that demand scalable architectures (e.g. bi-encoder retrieval with cross-encoder re-ranking in multi-stage pipelines). Bias mitigation has become critical; domain-specific models have been shown to yield fairer outcomes than off-the-shelf large language models. This review surveys recent (2021–2025) peer-reviewed work on CJM, covering algorithmic approaches (SVMs, ensemble trees, Siamese and cross-encoder transformers, GNNs, and hybrid recommenders), model architectures, input representations (resumes, job text, skill ontologies), and evaluation methods. We synthesize experimental findings from academic studies, discussing strengths and limitations of each approach, including accuracy, robustness, interpretability, and fairness. Finally, we highlight open challenges and directions for making CJM more transparent and equitable while maintaining scalability in practice.

Classical machine learning (ML) has proven itself successful in a vast range of chemical applications. These successes often only come after substantial effort in feature engineering to obtain an effective data representation for the model input. Graph neural networks (GNNs) limit this feature engineering by learning their own representation directly from molecular graphs. However, the lack of an expert-generated representation may put larger requirements on training data set size compared to classical ML approaches. GNNs often still require vertex and edge features to provide information that cannot be learned from graph connectivity alone, such as chemical elements and bond orders. In this work, we investigate if chemically inspired representations of these features (specifically chemical elements) can combat GNN data requirements and/or improve performance or generalizability of the models. To this end, nine representations of chemical elements were created that encode trends in elemental reactivity and behavior. We tested these representations across GNN model size, data set size and additional degrees of featurization, utilizing the prediction of molecular pKa values as a benchmark. Our results suggest that the models largely classify elements, converting all representations into a largely orthogonal representation used internally. This mechanism explains our findings of minimal differences in model performance and generalizability between elemental representation. However, slight performance benefits are observed with more orthogonal elemental representations, probably because they better match the internal representation of the models. These findings led us to investigate the use of representations with increased orthogonality in the form of (i) atom-typing (effectively including information on atomic neighbors of the graph vertices), which yielded modest performance gains, and (ii) similarity typing as a novel atomic representation (encoding for similarity between atom types), which provided further performance gains but with substantial increase in computational cost for data set preparation.

We present CrysMTM, a large-scale, multimodal dataset designed to benchmark temperature- and phase-sensitive machine learning models for crystalline materials. The dataset comprises approximately 30,000 atomistic samples covering the three primary polymorphs of Titanium dioxide--anatase, brookite, and rutile--each evaluated across a temperature spectrum ranging from cryogenic to ambient and elevated conditions. Each data entry integrates three complementary modalities: (1) three-dimensional atomic coordinates, (2) RGBA molecular visualizations, and (3) structured textual metadata encompassing geometric descriptors, local bonding environments, and phase transformation parameters. This multimodal structure enables both supervised and self-supervised learning across graph-based, image-based, and language-based architectures. CrysMTM supports rigorous evaluation of model robustness under thermal perturbations and crystallographic phase transitions. Baseline benchmarking across 18 models--including graph neural networks (GNNs), convolutional neural networks, and foundation models--reveals significant property-specific challenges. For example, bandgap predictions exhibit errors exceeding 25%, while volumetric expansion and atomic displacement estimations frequently deviate by more than 100%. Even state-of-the-art GNNs, which achieve an average in-distribution (ID) mean absolute percentage error of approximately 20%, show a threefold increase under out-of-distribution (OOD) thermal conditions. In contrast, a few-shot multimodal large language model reduces global prediction error from 96% to 23% and narrows the performance gap between ID and OOD cases to just four percentage points. These results highlight both the selective difficulty posed by temperature-sensitive geometric targets and the considerable room for innovation in model design. All dataset files, model implementations, and pretrained checkpoints are publicly available at https://github.com/KurbanIntelligenceLab/CrysMTM.

Outlier detection in temporal and spatial sequences via correlation analysis based on graph neural networks

Graph structured data is ubiquitous in daily life and scientific areas and has attracted increasing attention. Graph Neural Networks (GNNs) have been proved to be effective in modeling graph structured data and many variants of GNN architectures have been proposed. However, much human effort is often needed to tune the architecture depending on different datasets. Researchers naturally adopt Automated Machine Learning on Graph Learning, aiming to reduce human effort and achieve generally top-performing GNNs, but their methods focus more on the architecture search. To understand GNN practitioners' automated solutions, we organized AutoGraph Challenge at KDD Cup 2020, emphasizing automated graph neural networks for node classification. We received top solutions, especially from industrial technology companies like Meituan, Alibaba, and Twitter, which are already open sourced on GitHub. After detailed comparisons with solutions from academia, we quantify the gaps between academia and industry on modeling scope, effectiveness, and efficiency, and show that (1) academic AutoML for Graph solutions focus on GNN architecture search while industrial solutions, especially the winning ones in the KDD Cup, tend to obtain an overall solution (2) with only neural architecture search, academic solutions achieve on average 97.3% accuracy of industrial solutions (3) academic solutions are cheap to obtain with several GPU hours while industrial solutions take a few months' labors. Academic solutions also contain much fewer parameters.

Graph Neural Networks (GNNs) have achieved great success in dealing with non-Euclidean graph-structured data and have been widely deployed in many real-world applications. However, their effectiveness is often jeopardized under class-imbalanced training sets. Most existing studies have analyzed class-imbalanced node classification from a supervised learning perspective, they do not fully utilize the large number of unlabeled nodes in semi-supervised scenarios. We claim that the supervised signal is just the tip of the iceberg and a large number of unlabeled nodes have not yet been effectively utilized. In this work, we propose IceBerg, a debiased self-training framework to address the class-imbalanced and few-shot challenges for GNNs at the same time. Specifically, to figure out the Matthew effect and label distribution shift in self-training, we propose Double Balancing, which can largely improve the performance of existing baselines with just a few lines of code as a simple plug-and-play module. Secondly, to enhance the long-range propagation capability of GNNs, we disentangle the propagation and transformation operations of GNNs. Therefore, the weak supervision signals can propagate more effectively to address the few-shot issue. In summary, we find that leveraging unlabeled nodes can significantly enhance the performance of GNNs in class-imbalanced and few-shot scenarios, and even small, surgical modifications can lead to substantial performance improvements. Systematic experiments on benchmark datasets show that our method can deliver considerable performance gain over existing class-imbalanced node classification baselines. Additionally, due to IceBerg's outstanding ability to leverage unsupervised signals, it also achieves state-of-the-art results in few-shot node classification scenarios. The code of IceBerg is available at: https://github.com/ZhixunLEE/IceBerg.

Despite the significant potential of Foundation Models (FMs) in medical imaging, their application to prognosis prediction remains challenging due to data scarcity, class imbalance, and task complexity, limiting their clinical adoption. This study introduces the first structured benchmark to assess the robustness and efficiency of transfer learning strategies for FMs compared with convolutional neural networks (CNNs) in predicting COVID-19 patient outcomes from chest X-rays. The goal is to systematically compare fine-tuning strategies, classical and parameter-efficient, under realistic clinical constraints related to data scarcity and class imbalance, offering empirical guidance for AI deployment in clinical workflows. Four publicly available COVID-19 chest X-ray datasets were used, covering mortality, severity, and ICU admission, with varying sample sizes and class imbalances. CNNs pretrained on ImageNet and FMs pretrained on general or biomedical datasets were adapted using full fine-tuning, linear probing, and parameter-efficient methods. Models were evaluated under full-data and few-shot regimes using Matthews Correlation Coefficient (MCC) and Precision-Recall AUC (PR-AUC) with cross-validation and class-weighted losses. CNNs with full fine-tuning performed robustly on small, imbalanced datasets, while FMs with Parameter-Efficient Fine-Tuning (PEFT), particularly LoRA and BitFit, achieved competitive results on larger datasets. Severe class imbalance degraded PEFT performance, whereas balanced data mitigated this effect. In few-shot settings, FMs showed limited generalization, with linear probing yielding the most stable results. No single fine-tuning strategy proved universally optimal. CNNs remain dependable for low-resource scenarios, whereas FMs benefit from parameter-efficient methods when data are sufficient.

ABSTRACTAntimicrobial peptides (AMPs) are promising candidates to combat multidrug‐resistant pathogens. However, the high cost of extensive wet‐lab screening has made AI methods for identifying and designing AMPs increasingly important, with machine learning (ML) techniques playing a crucial role. AI approaches have recently revolutionised this field by accelerating the discovery of new peptides with anti‐infective activity, particularly in preclinical mouse models. Initially, classical ML approaches dominated the field, but recently there has been a shift towards deep learning (DL) models. Despite significant contributions, existing reviews have not thoroughly explored the potential of large language models (LLMs), graph neural networks (GNNs) and structure‐guided AMP discovery and design. This review aims to fill that gap by providing a comprehensive overview of the latest advancements, challenges and opportunities in using AI methods, with a particular emphasis on LLMs, GNNs and structure‐guided design. We discuss the limitations of current approaches and highlight the most relevant topics to address in the coming years for AMP discovery and design.

Graph Neural Networks (GNNs) excel in node classification tasks but often assume homophily, where connected nodes share similar labels. This assumption does not hold in many real-world heterophilic graphs. Existing models for heterophilic graphs primarily rely on pairwise relationships, overlooking multi-scale information from higher-order structures. This leads to suboptimal performance, particularly under noise from conflicting class information across nodes. To address these challenges, we propose HPGNN, a novel model integrating Higher-order Personalized PageRank with Graph Neural Networks. HPGNN introduces an efficient high-order approximation of Personalized PageRank (PPR) to capture long-range and multiscale node interactions. This approach reduces computational complexity and mitigates noise from surrounding information. By embedding higher-order structural information into convolutional networks, HPGNN effectively models key interactions across diverse graph dimensions. Extensive experiments on benchmark datasets demonstrate HPGNN’s effectiveness. The model achieves better performance than five out of seven state-of-the-art methods on heterophilic graphs in downstream tasks while maintaining competitive performance on homophilic graphs. HPGNN’s ability to balance multi-scale information and robustness to noise makes it a versatile solution for real-world graph learning challenges. Codes are available at https://github.com/streetcorner/HPGNN.

Electroencephalography (EEG) signals provide critical insights for applications in disease diagnosis and healthcare. However, the scarcity of labeled EEG data poses a significant challenge. Foundation models offer a promising solution by leveraging large-scale unlabeled data through pre-training, enabling strong performance across diverse tasks. While both temporal dynamics and inter-channel relationships are vital for understanding EEG signals, existing EEG foundation models primarily focus on the former, overlooking the latter. To address this limitation, we propose Graph-Enhanced EEG Foundation Model (GEFM), a novel foundation model for EEG that integrates both temporal and inter-channel information. Our architecture combines Graph Neural Networks (GNNs), which effectively capture relational structures, with a masked autoencoder to enable efficient pre-training. We evaluated our approach using three downstream tasks and experimented with various GNN architectures. The results demonstrate that our proposed model, particularly when employing the GCN architecture with optimized configurations, consistently outperformed baseline methods across all tasks. These findings suggest that our model serves as a robust foundation model for EEG analysis.