Abstract Background: Despite the recent success in Immune checkpoint blockade (ICB) therapy, limited patients benefit due to immune-related adverse events or treatment resistance, especially in solid tumors. To overcome these hurdles, understanding the molecular mechanisms behind treatment response is essential. However, the current lack of immune gene transcriptional programs (GTPs) hampers data-driven immunological discovery. Constructing immunity-specific knowledgebases with rigorously curated gene sets (irGSs) and rich immunological language can unlock new insights, enabling interpretation of high-throughput immune microenvironment profiling studies and fostering personalized, effective cancer treatments. Methods: We collected 83 BulkRNAseq datasets from the ImmuneSigDB. These datasets contain 1826 samples challenged with infections, cytokines or immunological perturbations of different kinds and magnitude, possessing yet-to-be discovered immune functions that lie beneath the transcriptomic profiles. Using non-negative matrix factorization (NMF), we identified gene sets with coordinated expression. We used CITE-seq and selected scRNAseq to annotate the immunological functions of each gene set and validated the clinical utilities of these gene sets from different aspects using data from Cancer Genome Atlas Program (TCGA) pan-cancer, ICB cohorts and a 10X Genomics Visium FFPE Human Breast Cancer spatial slide. Results: We presented 19 lymphoid-data derived and 9 myeloid-data derived gene sets (irGSs), encompassing a diverse array of immune functions. Through extensive validation using BulkRNAseq, scRNAseq and spatial transcriptomics data, we revealed six irGSs-defined pan-cancer microenvironment subtypes in TCGA with significantly distinct survival patterns regardless of cancer types. Moreover, irGSs are associated with T cell exhaustion phenotypes, which in turn well predicted (classification accuracy of 72%) Nivolumab response using 104 treatment-naïve melanoma patients. Lastly, irGSs simultaneously isolated out tumor regions and highlighted immune infiltration with high specificity in breast cancer spatial transcriptomics H&E image. Conclusions: These pioneering gene sets, originally derived from non-cancerous experiments, hold tremendous promise for cancer research across diverse contexts. Similarities between cancerous and sepsis immune microenvironments underscore their wide applicability. By studying gene set activities, immunologists can gain profound insights into cancer survival drivers, unravel the intricacies of ICB treatment mechanisms, and potentially conquer therapeutic resistance. These translational utilities herald a transformative era in cancer immunotherapy and open new frontiers in the fight against cancer. Citation Format: Shan He, Vakul Mohanty, Matthew Gubin, Hind Rafei, Rafet Basar, Merve Dede, Xianli Jiang, Yukun Tan, Maura Gillison, Katayoun Rezvani, Weiyi Peng, Ken Chen. Elucidating immune-related gene transcriptional programs via factorization of large-scale RNA profiles [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 873.