Rubberized asphaltic materials have been frequently combined with warm-mix asphalt technologies to tackle the issues of high energy consumptions and emissions during construction. Effective and accurate characterization of binder properties is conducive to the improvement of long-term pavement performance. The current study aims to quantify the effects of rubber content and warm-mix additives on rutting and thermal cracking performance of crumb rubber-modified bitumen (CRMB), and explore the rubber and additives modification mechanisms and their impacts on the binder performance. CRMBs containing different rubber contents and warm-mix additives after long-term aging were subject to multiple stress creep and recovery (MSCR) tests and low-temperature frequency sweep tests using a dynamic shear rheometer (DSR) with 4-mm loading plate to investigate the high- and low-temperature performance, respectively. Rheological tests were also conducted on the bitumen and rubber phases of CRMB to understand the rubber modification mechanism. Results show that CRMB binders have superior rutting and thermal cracking resistance due to rubber modification. The improvement of high- or low-temperature performance is more prominent at higher rubber concentrations. The effects of warm-mix additives on the rutting and thermal cracking performance are different. Generally, the wax-based additive improves the rutting resistance but negatively affects the low-temperature performance. In contrast, the chemical-based additive has an opposite effect except for the high-temperature performance of neat bitumen. The stiffening of the bitumen phase and the contribution of swollen rubber particles in the bitumen matrix together contribute to the peculiar viscoelastic response of CRMB, i.e., stiffer/softer and more elastic at high/low temperatures. This modification mechanism explains the superior rutting and thermal cracking performance of CRMB.