The evaluation of treatment response in patients with gliomas is performed using the Response Assessment in Neuro-Oncology (RANO) criteria. These criteria are based on cerebral magnetic resonance imaging (MRI), steroid use, and neurological function. However, a standardized tool for evaluating neurological function was lacking. We compared changes in the National Institute of Health Stroke Scale (NIHSS) to changes in the RANO categories to determine the relationship between clinical and neuroradiological findings. We reviewed data on all adult patients with supratentorial gliomas WHO grade II-IV who were treated at the Cantonal Hospital St. Gallen from 2008 to 2015. The NIHSS was performed prospectively at baseline and at 3-month intervals simultaneously to MRI. Associations between changes in the NIHSS and RANO categories were assessed using the Stuart-Maxwell test. Our cohort consisted of 61 patients from which 471 observations were analyzed. The most common histological diagnosis was glioblastoma (49.2%). In total, 74% of RANO categories and 81% of the NIHSS scores remained stable on follow-up. Statistically, contemporaneous changes in the RANO category did not correlate with changes in the NIHSS (P < .0001). The application of the NIHSS is easy and feasible in the heterogeneous population of glioma patients. In our cohort, the RANO categories did not reflect contemporaneous changes in the NIHSS. A validated clinical outcome measure with a well-defined minimal clinically important difference is warranted in neuro-oncological research and clinical practice.