Abstract Functional gene expression signatures (FGES) are widely used as biomarkers in cancer diagnostics as descriptive and predictive models for treatment selection. Since FGES scores are based solely on gene expression profiles, they can be applied to data from bulk and single-cell RNA-seq or gene expression microarrays. However, FGES development and implementation has several challenges including technical limitations of calculating FGES scores based on ranked gene expressions such as background noise and gene cross-correlations and difficulty in assessing the biological relevance of FGES. Here, we developed an FGES validation pipeline that addressed these limitations by examining publicly available and internally-developed FGES. The pipeline was developed by applying a modified single-sample gene set enrichment analysis (ssGSEA) to a manually curated database of more than 50,000 RNA-Seq samples, including sorted cells, cell lines, and tumor and normal tissue biopsies. We defined technical and biological criteria to validate signature quality: 1) single gene expression noise distribution minimum is greater than zero; 2) cross-correlation of genes comprising a signature is positive; 3) specificity, whereby a FGES differentiates a group of interest and has a minimal overlap with other unrelated signatures and molecular pathways; 4) generalizability, requiring FGES score to differentiate a group of interest across datasets of different origin, cell type, or diagnosis. We tested these criteria on FGES that describe cellular processes such as epithelial-mesenchymal transition (EMT) and senescence as well as FGES that can delineate cell types such as macrophages. We were able to better distinguish 12 different cell types using the corresponding cell type-specific FGES with significantly higher ssGSEA scores (p-adj ≤ 0.00004 in 99.9% cases) than 911 analogous MSigDB gene signatures. Our EMT FGES, tuned specifically for melanoma, exhibited higher precision than the MSigDB Hallmark EMT gene set (p < 0.001) for predicting histological pigment scores. The senescence FGES also accurately distinguished senescent from non-senescent cells in datasets of differing cell origin and data acquisition methods, demonstrating signature generalizability. Finally, using macrophage-describing FGES, we showed that specificity of a publicly available gene set can be significantly improved (p-adj = 0.00001) by tuning it according to the aforementioned criteria. Taken together, our proposed workflow enables better assessment of the technical quality and biological adequacy of FGES for their further use in transcriptomic analyses and computational models of complex biological processes and tumorigenesis. Citation Format: Elena Ocheredko, Nadezhda Lukashevich, Maria Savchenko, Sofia Kust, Syune Ambaryan, Dmitry Tabakov, Maria Sorokina, Linda Balabanian, Kushal Suryamohan, Anastasiya Zotova, Nathan Fowler, Aleksander Bagaev. Development and refinement of functional gene expression signatures as a computational tool for comprehensive characterization of transcriptomic data [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 6538.