COVID-19 can cause severe neurological symptoms, but the underlying pathophysiological mechanisms are unclear. Here, we interrogated the brain stems and olfactory bulbs in postmortem patients who had COVID-19 using imaging mass cytometry to understand the local immune response at a spatially resolved, high-dimensional, single-cell level and compared their immune map to non-COVID respiratory failure, multiple sclerosis, and control patients. We observed substantial immune activation in the central nervous system with pronounced neuropathology (astrocytosis, axonal damage, and blood-brain-barrier leakage) and detected viral antigen in ACE2-receptor-positive cells enriched in the vascular compartment. Microglial nodules and the perivascular compartment represented COVID-19-specific, microanatomic-immune niches with context-specific cellular interactions enriched for activated CD8+ T cells. Altered brain T-cell-microglial interactions were linked to clinical measures of systemic inflammation and disturbed hemostasis. This study identifies profound neuroinflammation with activation of innate and adaptive immune cells as correlates of COVID-19 neuropathology, with implications for potential therapeutic strategies.