Abstract Immune phenotypes (IP), defined by the tumor-infiltrating lymphocyte (TIL) distribution within the tumor microenvironment (TME), is prognostic and predictive of treatment response. Here, machine learning (ML) models that characterize the TME were deployed in non-small cell lung cancer (NSCLC) and head and neck squamous cell carcinoma (HNSCC) to exhaustively label TILs directly from hematoxylin and eosin (H&E)-stained whole slide images (WSI), compute IP and quantify TIL distribution, tasks which are manually untenable. ML-powered TME models (PathAI, Boston, MA; commercially available as PathExplore™) for NSCLC and HNSCC that quantify tissue regions (e.g. tumor, epithelium, and stroma) and cells (e.g. lymphocytes) in H&E-stained WSI were deployed on 126 NSCLC and 103 HNSCC commercial samples. Ground truth CD3 and CD8 scores were obtained from paired immunohistochemically-stained WSI (HALO, Indica Labs, Albuquerque, NM). Inflamed, desert, or excluded IPs (based on data-driven cutoffs) were inferred from the mean TIL density in epithelium and stroma (slide-level IP, sIP), and from the fraction of “hot” patches in all 100um x 100um patches tiling the tumor and stroma areas (patch-level IP, pIP). TIL spatial distribution was measured by the Morisita-Horn index that computes the patch-wise overlap between TILs and cancer cells (MHI, ranging from 0 to 1), and epithelial-stromal interface distance index (EDI) indicating the hot stromal patch bias toward the epithelium (-EDI) or stroma (+EDI). ML-predicted TIL densities highly correlated with ground truth CD3+ and CD8+ cell densities (NSCLC/HNSCC CD3, CD8: tumor Pearson r= 0.88/0.85, 0.74/0.74; epithelium r= 0.76/0.71, 0.88/0.62; stroma r= 0.74/0.87, 0.61/0.76). High s- and pIP agreement was seen (NSCLC 93% and HNSCC 83%); 6/26 discordant cases were driven by TIL hotspots with high density (116%-310% of the cutoff) but few (median = 26%) hot patches in the epithelium. MHI was higher for the inflamed vs excluded IP (p < 0.0001 for NSCLC and HNSCC) and intra-group variability was high (NSCLC/HNSCC inflamed: 0.66±0.11/0.31±0.14, excluded: 0.51±0.12/0.26±0.17, desert: 0.48±0.09/0.28±0.17; mean±std). EDI was lowest and negative in inflamed IP but near zero in desert and excluded IP (NSCLC/HNSCC inflamed: -43±4um/-184±19um, excluded: 1.6±4um/-41±13um, desert: 18±4um/-14±13; mean±sem). Excluded and desert IPs had roughly equal + and - EDI cases (NSCLC +/-: 57/46; HNSCC +/-: 39/29). ML-powered IP prediction using TIL distribution enables accurate and rapid profiling of the TME using routine histopathology. pIPs were concordant with sIPs and highlight TIL heterogeneity. Spatial markers (MHI and EDI) reveal differences between IP classes and intra-group heterogeneity relevant for drug discovery and patient stratification. Investigating prognostic associations of these markers is a promising direction for future studies. Citation Format: Nhat Le, Bahar Rahsepar, Jennifer Hipp, Jake Conway, Ylaine Gerardin, Emma Krause, Ciyue Shen, Raymond Biju, Michael Nercessian, Nicholas Indorf, Sandrine Degryse, Miles Markey, Victoria Mountain, Pranjal Vaidya, William Wijaya, Aditee Shrotre, Patrick Caplazi, David Inzunza, Joann Palma, Erik Huntzicker, Catherine Tribouley, Diana Chen, Raluca Prediou, Francine Chen, Kevin Kolahi. ML quantification of tumor-Infiltrating lymphocytes distinguishes immune-phenotypes and reveals phenotypic heterogeneity [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 905.