Understanding the long-term performance deterioration trends and mechanisms of asphalt pavement is crucial for effective maintenance strategies. This study characterizes and correlates the multi-scale performance deterioration of a 14-year asphalt pavement. Air void measurements, indirect tensile (IDT) fatigue testing, Fourier transform infrared spectroscopy (FTIR), and dynamic shear rheometer (DSR) testing were conducted on pavement cores and recovered binder. Multiple regression analysis was then performed on various performance indicators. Laboratory results indicate that the chemical composition and viscoelastic properties of SBS-modified binders evolve rapidly in the first few years, followed by a relatively stable aging rate. After 14 years, the mechanical and rheological properties of lower-layer mixtures deteriorate to a similar degree as the surface layer. Correlation analysis revealed that the residual strength of the mixture is more influenced by air voids, while reductions in fatigue life are primarily driven by binder aging. These findings highlight the necessity of applying preventive maintenance within the first 3–5 years to rejuvenate the surface asphalt and rehabilitate both the surface and underlying layers after long-term service.