In response to stimuli, the immediate early gene product Arc can acutely down-regulate synaptic strength by removing AMPA receptors (AMPARs) from synapses and thus regulate synaptic plasticity. How Arc, a scaffold protein, can specifically facilitate synaptic removal of AMPARs is unknown. We found that Arc directly antagonizes with PSD-95 in binding to TARPs, which are the auxiliary subunits of AMPARs. Arc, in a highly concentration-sensitive manner, acutely disperses TARPs from the postsynaptic density (PSD) condensate formed via phase separation. TARPs with the Ser residue in the "P-S-Y"-motif of its tail phosphorylated are completely refractory from being dispersed by Arc, suggesting that Arc cannot displace AMPARs from PSDs in active synapses. Conversely, strengthening the interaction between Arc and TARPs enhances Arc's capacity in weakening synapses. Thus, Arc can specifically and effectively modulate synaptic AMPAR clustering via modulating PSD phase separation. Our study further suggests that activity-dependent, bi-directional modulation of PSD condensate formation/dispersion represents a general regulatory mechanism for synaptic plasticity.