Cognitive impairment is common in multiple sclerosis and negatively impacts quality of life. Cognitive status has yet to be described in people with severe progressive multiple sclerosis, in whom conventional neuropsychological testing is exceptionally difficult. The objective for the study was to characterize cognitive performance in severe progressive multiple sclerosis and compare them with age-, sex- and disease duration-matched less disabled people with multiple sclerosis using a specifically developed auditory, non-motor test of attention/cognitive processing speed-Auditory Test of Processing Speed. Also, we aimed to determine the relationship between cognitive performance and MRI-based outcomes in these matched cohorts. The Comprehensive Assessment of Severely Affected Multiple Sclerosis study was carried out at the University at Buffalo and the Boston Home, a skilled nursing facility in Dorchester, MA. Inclusion criteria were age 30-80 years and expanded disability status scale 3.0-6.5 for community-dwelling and 7.0-9.5 for skilled nursing facility people with multiple sclerosis. The cognitive assessment was performed using the Brief International Cognitive Assessment for Multiple Sclerosis consisting of Symbol Digit Modalities Test, Brief Visuospatial Memory Test-Revised, California Verbal Learning Test-2nd edition along with Auditory Test of Processing Speed, Paced Auditory Serial Addition Test-3 second and Controlled Oral Word Association Test. MRI scans were retrospectively collected and analysed for lesion and volumetric brain measurements. The rate of completion and performance of the cognitive tests was compared between the groups, and the relationship with MRI measures was determined using sex, age and years of education-adjusted linear regression models. Significantly greater percentage of the severe multiple sclerosis group completed Auditory Test of Processing Speed when compared with the current gold standard of Symbol Digit Modalities Test (93.2% versus 65.9%). Severe progressive multiple sclerosis had worse cognitive performance in all cognitive domains with greatest differences for cognitive processing speed (Symbol Digit Modalities Test > Paced Auditory Serial Addition Test-3 second > Auditory Test of Processing Speed, Cohen's d < 2.13, P < 0.001), learning and memory (Cohen's d < 1.1, P < 0.001) and language (Controlled Oral Word Association Test with Cohen's d = 0.97, P < 0.001). Multiple cognitive domains were significantly associated with lower thalamic (standardized β < 0.419, P < 0.006) and cortical (standardized β < 0.26, P < 0.031) volumes. Specially designed (auditory) cognitive processing speed tests may provide more sensitive screening of cognitive function in severe progressive multiple sclerosis. The cognitive profile of severe multiple sclerosis is proportional to their physical outcomes and best explained by decreased grey matter volume.