To improve the toughness of transportation infrastructure and develop asphalt composite materials with both elasticity and toughness, this paper prepared Buton rock asphalt (BRA)/Bis[2-(2-butoxyethoxy)ethyl] adipate BBOEA composite modified asphalt. The evolution of high and low-temperature performance of the composite modified asphalt was explored through key asphalt indicators, dynamic shear rheology, and bending beam rheometer (BBR) tests. The performance improvement mechanism of the asphalt was investigated using Fourier transform infrared spectroscopy (FTIR). The addition of Buton rock asphalt effectively improved the high-temperature performance of the asphalt, raising the high-temperature PG grade from 64 to 70 and enhancing resistance to permanent deformation. The inclusion of 1.5 % BBOEA further enhanced low-temperature performance, reducing the stiffness modulus of BRA modified asphalt from 365 MPa to 260 MPa, a decrease of 26.9 %. This improvement is attributed to the polar adipate functional groups in BBOEA, which promote hydrogen bonding and dipole–dipole interactions with polar groups in the resins, reducing micelle size and asphalt rigidity. With the high-temperature performance remaining robust and low-temperature flexibility and strain relaxation greatly improved, the optimal BBOEA content is recommended at 1.5 % of the asphalt mass.