Abstract LB253: Crafted experiments to evaluate feature selection methods for single cell RNA-seq data

Abstract

Abstract Single-cell RNA sequencing (scRNA-seq) technologies have revolutionized our understanding of cellular compositions and tumor behavior at the single-cell level, and have been widely used in many biomedical applications, such as identifying and characterizing novel cell types/states. While numerous computational methods have been developed for analyzing scRNA-seq data, benchmarking various analytical methods remains a key challenge for two reasons. First, there is a lack of ground truth datasets where unambiguous cell type identities are known using external information. Second, simulation of scRNA-seq data tends to be overly artificial and simplistic, which may not well mimic the underlying biological complexity as well as high level of technical variability. Here, we propose the use of crafted experiments, a new approach based upon perturbing signals in a real dataset for comparing different scRNA-seq analytical methodologies. We demonstrate the effectiveness of crafted experiments in the context of a novel univariate distribution-oriented suite of feature selection methods, called GOF (Goodness of Fit). We show that GOF more frequently selects features that robustly identify crafted artificial clusters in crafted experiments and achieves similar performance compared to the field standard method using real datasets. Importantly, we show the use of crafted experiments for identifying the contexts in which each method performs the best. Crafted experiments offer valuable comparisons of scRNA-seq analysis methods, and the crafted datasets generated from this study provide a useful resource for the single-cell community. Citation Format: Siyao Liu, David Corcoran, Susana Garcia-Recio, Charles Perou, J.S. Marron. Crafted experiments to evaluate feature selection methods for single cell RNA-seq data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 2 (Late-Breaking, Clinical Trial, and Invited Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(7_Suppl):Abstract nr LB253.