Understanding how sleep affects the glymphatic system and human brain networks is crucial for elucidating the neurophysiological mechanism underpinning aging-related memory declines. We analyzed a multimodal dataset collected through magnetic resonance imaging (MRI) and polysomnographic recording from 72 older adults. A proxy of the glymphatic functioning was obtained from the Diffusion Tensor Image Analysis along the Perivascular Space (DTI-ALPS) index. Structural and functional brain networks were constructed based on MRI data, and coupling between the two networks (SC-FC coupling) was also calculated. Correlation analyses revealed that DTI-ALPS was negatively correlated with sleep quality measures [e.g., Pittsburgh Sleep Quality Index (PSQI) and apnea-hypopnea index]. Regarding human brain networks, DTI-ALPS was associated with the strength of both functional connectivity (FC) and structural connectivity (SC) involving regions such asthe middle temporal gyrus and parahippocampal gyrus, as well as with the SC-FC coupling ofrich-club connections. Furthermore, we found that DTI-ALPS positively mediated the association between sleep quality and rich-club SC-FC coupling. The rich-club SC-FC coupling further mediated the association between DTI-ALPS and memory function in good sleepers but not in poor sleepers. The results suggest a disrupted glymphatic-brain relationship in poor sleepers, which underlies memory decline. Our findings add important evidence that sleep quality affects cognitive health through the underlying neural relationships and the interplay between the glymphatic system and multimodal brain networks.