Background: Identifying magnetic resonance imaging (MRI) markers in myelin-oligodendrocytes-glycoprotein antibody-associated disease (MOGAD), neuromyelitis optica spectrum disorder-aquaporin-4 positive (NMOSD-AQP4) and multiple sclerosis (MS) is essential for establishing objective outcome measures. Objectives: To quantify imaging patterns of central nervous system (CNS) damage in MOGAD during the remission stage, and to compare it with NMOSD-AQP4 and MS. Methods: 20 MOGAD, 19 NMOSD-AQP4, 18 MS in remission with brain or spinal cord involvement and 18 healthy controls (HC) were recruited. Volumetrics, lesions and cortical lesions, diffusion-imaging measures, were analysed. Results: Deep grey matter volumes were lower in MOGAD (p = 0.02) and MS (p = 0.0001), compared to HC and were strongly correlated with current lesion volume (MOGAD R = −0.93, p < 0.001, MS R = −0.65, p = 0.0034). Cortical/juxtacortical lesions were seen in a minority of MOGAD, in a majority of MS and in none of NMOSD-AQP4. Non-lesional tissue fractional anisotropy (FA) was only reduced in MS (p = 0.01), although focal reductions were noted in NMOSD-AQP4, reflecting mainly optic nerve and corticospinal tract pathways. Conclusion: MOGAD patients are left with grey matter damage, and this may be related to persistent white matter lesions. NMOSD-AQP4 patients showed a relative sparing of deep grey matter volumes, but reduced non-lesional tissue FA. Observations from our study can be used to identify new markers of damage for future multicentre studies.