Although its incidence is relatively low, delayed-onset post-stroke cognitive decline (PSCD) may offer valuable insights into the "vascular contributions to cognitive impairment and dementia," particularly concerning the roles of vascular and neurodegenerative mechanisms. We postulated that the functional segregation observed during post-stroke compensation could be disrupted by underlying amyloid pathology or cerebral small vessel disease (cSVD), leading to delayed-onset PSCD. Using a prospective stroke registry, we identified patients who displayed normal cognitive function at baseline evaluation within a year post-stroke and received at least one subsequent assessment. Patients suspected of pre-stroke cognitive decline were excluded. Decliners [defined by a decrease of ≥3 Mini-Mental State Examination (MMSE) points annually or an absolute drop of ≥5 points between evaluations, confirmed with detailed neuropsychological tests] were compared with age- and stroke severity-matched non-decliners. Index-stroke MRI, resting-state functional MRI, and 18F-florbetaben PET were used to identify cSVD, functional network attributes, and amyloid deposits, respectively. PET data from age-, sex-, education-, and apolipoprotein E-matched stroke-free controls within a community-dwelling cohort were used to benchmark amyloid deposition. Among 208 eligible patients, 11 decliners and 10 matched non-decliners were identified over an average follow-up of 5.7 years. No significant differences in cSVD markers were noted between the groups, except for white matter hyperintensities (WMHs), which were strongly linked with MMSE scores among decliners (rho = -0.85, p < 0.01). Only one decliner was amyloid-positive, yet subthreshold PET standardized uptake value ratios (SUVR) in amyloid-negative decliners inversely correlated with final MMSE scores (rho = -0.67, p = 0.04). Decliners exhibited disrupted modular structures and more intermingled canonical networks compared to non-decliners. Notably, the somato-motor network's system segregation corresponded with the decliners' final MMSE (rho = 0.67, p = 0.03) and was associated with WMH volume and amyloid SUVR. Disruptions in modular structures, system segregation, and inter-network communication in the brain may be the pathophysiological underpinnings of delayed-onset PSCD. WMHs and subthreshold amyloid deposition could contribute to these disruptions in functional brain networks. Given the limited number of patients and potential residual confounding, our results should be considered hypothesis-generating and need replication in larger cohorts in the future.