Neurocritically ill patients are at high risk for developing delirium, which can worsen the long-term outcomes of this vulnerable population. However, existing delirium assessment tools do not account for neurologic deficits that often interfere with conventional testing and are therefore unreliable in neurocritically ill patients. We aimed to determine the accuracy and predictive validity of the Fluctuating Mental Status Evaluation (FMSE), a novel delirium screening tool developed specifically for neurocritically ill patients. Prospective validation study. Neurocritical care unit at an academic medical center. One hundred thirty-nine neurocritically ill stroke patients (mean age, 63.9 [sd, 15.9], median National Institutes of Health Stroke Scale score 11 [interquartile range, 2-17]). None. Expert raters performed daily Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition-based delirium assessments, while paired FMSE assessments were performed by trained clinicians. We analyzed 717 total noncomatose days of paired assessments, of which 52% (n = 373) were rated by experts as days with delirium; 53% of subjects were delirious during one or more days. Compared with expert ratings, the overall accuracy of the FMSE was high (area under the curve [AUC], 0.85; 95% CI, 0.82-0.87). FMSE scores greater than or equal to 1 had 86% sensitivity and 74% specificity on a per-assessment basis, while scores greater than or equal to 2 had 70% sensitivity and 88% specificity. Accuracy remained high in patients with aphasia (FMSE ≥ 1: 82% sensitivity, 64% specificity; FMSE ≥ 2: 64% sensitivity, 84% specificity) and those with decreased arousal (FMSE ≥ 1: 87% sensitivity, 77% specificity; FMSE ≥ 2: 71% sensitivity, 90% specificity). Positive FMSE assessments also had excellent accuracy when predicting functional outcomes at discharge (AUC, 0.86 [95% CI, 0.79-0.93]) and 3 months (AUC, 0.85 [95% CI, 0.78-0.92]). In this validation study, we found that the FMSE was an accurate delirium screening tool in neurocritically ill stroke patients. FMSE scores greater than or equal to 1 indicate "possible" delirium and should be used when prioritizing sensitivity, whereas scores greater than or equal to 2 indicate "probable" delirium and should be used when prioritizing specificity.