Fly ash from municipal solid waste incineration (MSWIFA) contains leachable heavy metals (HMs), and the environmental risk of contained HMs is an important concern for its safe treatment and disposal. This paper presents a dynamic leaching test of fly ash-based cementitious materials containing arsenic (FCAC) in three particle sizes based on an innovative simulation of two acid rainfall conditions to investigate the long-term stability of FCAC under acid rain conditions. As well as semi-dynamic leaching test by simulating FCAC in three scenarios. Furthermore, the long-term stability risk of FCAC is evaluated using a sequential extraction procedure (SEP) and the potential risk assessment index. Results showed that the Al3+ in the FCAC dissolved and reacted with the OH− in solution to form Al(OH)3 colloids as the leaching time increased. Moreover, the oxidation of sulfide minerals in the slag produced oxidants, such as H2SO4 and Fe2(SO4)3, which further aggravated the oxidative dissolution of sulfides, thereby resulting in an overall decreasing pH value of the leachate. In addition, due to the varying particle sizes of the FCAC, surface area size, and adsorption site changes, the arsenic leaching process showed three stages of leaching characteristics, namely, initial, rapid, and slow release, with a maximum leaching concentration of 2.42 mg/L, the cumulative release of 133.78 mg/kg, and the cumulative release rate of 2.32%. The SEP test revealed that the reduced state of HMs in the raw slag was lowered substantially, and the acid extractable state and residual state of HMs were increased, which was conducive to lessening the risk of FCAC. Overall, the geological polymerization reaction of MSWIFA is a viable and promising solution to stabilize mining and industrial wastes and repurpose the wastes into construction materials.