Verbal memory decline is a common complaint of patients with severe asymptomatic stenosis of the internal carotid artery (aICS). Previous publications explored the associations between verbal memory decline and altered functional connectivity (FC) after aICS. Patients with severe aICS may show reduced perfusion in the ipsilateral territory and redistribution of cerebral blood flow to compensate for the deficient regions, including expansion of the posterior and contralateral ICA territories via the circle of Willis. However, aICS-related FC changes in anterior and posterior territories and the impact of the sides of stenosis were less explored. This study aims to investigate the altered FC in anterior and posterior circulation territories of patients with left or right unilateral aICS and its association with verbal memory decline. We enrolled 15 healthy controls (HCs), 22 patients with left aICS (aICSL), and 33 patients with right aICS (aICSR) to receive fMRI, Mini-Mental State Examination (MMSE), the Digit Span Test (DST), and the 12-item Chinese version of Verbal Learning Tests. We selected brain regions associated with verbal memory within anterior and posterior circulation territories. Territory-related FC alterations and verbal memory decline were identified by comparing the aICSL and aICSR groups with HC groups (P < .05, corrected for multiple comparisons), respectively. Furthermore, the association between altered FC and verbal memory decline was tested with the Pearson correlation analysis. Compared with HCs, patients with aICSL or aICSR had significant impairment in delayed recall of verbal memory. Decline in delayed recall of verbal memory was significantly associated with altered FC between the right cerebellum and right middle temporal pole in the posterior circulation territory (r = 0.40, P = .03) in the aICSR group and was significantly associated with altered FC between the right superior medial frontal gyrus and left lingual gyrus in the anterior circulation territory (r = 0.56, P = .01) in the aICSL group. Patients with aICSL and aICSR showed different patterns of FC alterations in both anterior and posterior circulation territories, which suggests that the side of aICS influences the compensatory mechanism for decline in delayed recall of verbal memory.