The chemo-rheological properties of crumb rubber modified bitumen are always unstable due to the mutable and uncontrollable swelling-degradation degree of crumb rubber in bitumen matrix. The study aimed at exploring the continuous swelling and degradation behaviors of crumb rubber modified bitumen (CRMB) considering the influence of rubber size through monitoring the dynamic viscosity changes of CRMB binders. Moreover, the synergistic effects of swelling-degradation degree and rubber size on the chemical and rheological properties of CRMB were investigated. The results revealed that the rubber size significantly influenced the swelling and degradation behaviors of CRMB. The reduction of rubber size shortened the equilibrium swelling and degradation time, while increased the related viscosity dramatically. Moreover, during the degradation process, the decrease of rubber size could accelerate the continuous swelling rate, increase the maximum viscosity and reduce the continuous swelling time of CRMB. Meanwhile, the high swelling degree and large rubber size were beneficial to enhance the high temperature properties, while the CRMB binder with high degradation degree showed the better low-temperature property, workability and wider Newtonian flow region. Furthermore, the degradation degree promoted the formation of free hydroxide groups, aldehydes, carboxylic acids and esters, while the swelling process increased the average molecular weight of whole liquid phase in CRMB binder. The outputs from this fundamental study are beneficial to provide the guidance to preparation conditions optimization of CRMB binders with different viscous property standards.