The bond between aggregate and asphalt binder depends on many factors, such as chemistry of materials, nature of aggregates, and quality of asphalt binder. The present study evaluates the effects of recycled asphalt pavement (RAP) on the bonding behavior of asphalt binder for acidic (granite and quartzite) and basic (basalt and limestone) aggregates. AC30 binder was blended with different proportions of RAP binder (15%, 25%, and 40%). The interfacial energy and bond strength of 16 combinations (four asphalt binders × four aggregates) of aggregate-asphalt binder system were evaluated based on a surface free energy (SFE) approach and binder bond strength (BBS) test, respectively. One of the strong motivations of the present study was to evaluate how interfacial energy parameters calculated based on the SFE of an aggregate-asphalt binder system may correlate with the outcome obtained from a BBS test. The results showed that AC30 binder modified with RAP binder had a good performance for basic aggregates compared with acidic aggregates. Basic aggregates (limestone and basalt) showed cohesive failure before and after conditioning, indicating minimal effects of water on interfacial bond strength. However, acidic aggregates (granite and quartzite) showed cohesive failure in dry conditions and adhesive failure after wet conditioning, implying detrimental effects of water on interfacial bond strength for these aggregates. The failure pattern of aggregates in cohesion was supported by a reduction in cohesion energy of asphalt binder after the addition of RAP binder. Correlation between interfacial energy parameters (obtained from SFE) and BBS strength depends on aggregate types and failure patterns.