Moisture damage is a prominent problem of asphalt pavements. The bond strength between asphalt and aggregates is a crucial factor that influences the capability of asphalt to resist moisture-induced damage. In this study, a binder bond strength (BBS) test was conducted to evaluate the effects of various modifiers and additives of different amounts on bond strength between asphalt and aggregates. Furthermore, the influence of styrene–butadiene–styrene (SBS) on adhesion behavior of asphalt binder was investigated through a gel permeation chromatography (GPC) test. Finally, the results of the BBS test were compared with the findings obtained from a Hamburg wheel-tracking device (HWTD) test, which reflected the moisture sensitivity of mixtures under wet conditions. Results indicated that gilsonite, high-density polyethylene, and polyphosphoric acid improved the bond strength of the base asphalt; SBS had no positive effects on asphalt adhesion properties; and SBS at a low amount reduced the bond strength. Ethylene bis-stearamide wax, crumb rubber, terminal blend (TB) rubber powder, and compound modifier TB rubber powder plus SBS decreased the bond strength. The GPC test results showed that SBS possibly did not actively contribute to the formation of bond strength between asphalt and aggregates. Test data for BBS and HWTD tests under wet conditions confirmed that there was no discernible correlation between these two tests when adhesion properties of modified asphalts were evaluated. However, the results of the BBS test were in accordance with those of the HWTD test when the adhesion of asphalt with different amounts of the same modifier and the mixture resistance to water damage were evaluated.