Altered glycosylation profiles have been correlated with potential drug targets in various diseases, including Alzheimer’s disease (AD). In this area, the linkage between bisecting N-acetylglucosamine (GlcNAc), a product of N-acetylglucosaminyltransferase III (GnT-III), and AD has been recognized, however, our understanding of the cause and the causative role of this aberrant glycosylation in AD are far from completion. Moreover, the effects and mechanisms of glycosylation-targeting interventions on memory and cognition, and novel targeting strategies are worth further study. Here, we showed the characteristic amyloid pathology-induced and age-related changes of GnT-III, and identified transcription factor 7-like 2 as the key transcription factor responsible for the abnormal expression of GnT-III in AD. Upregulation of GnT-III aggravated cognitive dysfunction and Alzheimer-like pathologies. In contrast, loss of GnT-III could improve cognition and alleviate pathologies. Furthermore, we found that an increase in bisecting GlcNAc modified ICAM-1 resulted in impairment of microglial responses, and genetic inactivation of GnT-III protected against AD mechanistically by blocking the aberrant glycosylation of ICAM-1 and subsequently modulating microglial responses, including microglial motility, phagocytosis ability, homeostatic/reactive state and neuroinflammation. Moreover, by target-based screening of GnT-III inhibitors from FDA-approved drug library, we identified two compounds, regorafenib and dihydroergocristine mesylate, showing pharmacological potential leading to modulation of aberrant glycosylation and microglial responses, and rescue of memory and cognition deficits.