Tumor-infiltrating immune cells have been correlated with prognosis for patients treated with immune checkpoint inhibitor (ICI) treatment of various cancers. However, no robust biomarker has been described to predict treatment response yet. We hypothesized that the activation potency of circulating T cells may predict response to ICI treatment. An exploratory analysis was conducted to investigate the association between the response to immune checkpoint inhibition (ICI) combined with stereotactic radiotherapy (SBRT) and the potency of circulating T cells to be activated. Blood-derived lymphocytes from 14 patients were stimulated ex vivo with, among others, Staphylococcal enterotoxin B (SEB) and compared to healthy controls (HCs). Patients were grouped into responders (>median progression free survival (PFS)) and non-responders (<median PFS). The expression of the T cell activation marker CD69 and intracellular cytokines (IL-2, IFNγ, TNFα) in both CD4+ and CD8+ T cells in response to stimulation was measured using flow cytometry. In addition, serum levels of BAFF, IFNγ, and IL-2 receptor (sIL-2R) were measured by Luminex. At baseline, a higher percentage of activated CD8+ T cells (15.8% vs. 3.5% (p = <0.01)) and IL-2+CD69+CD8+ T cells (8.8% vs. 2.9% (p = 0.02)) was observed in responders compared to non-responders upon ex vivo stimulation with SEB. The concurrently measured serum cytokine levels were not different between responders and non-responders. Baseline blood CD8+ T cell activation potency, measured by intracellular cytokine production after ex vivo stimulation, is a potential biomarker to discriminate responders from non-responders to SBRT combined with ICI.