Drug-target affinity (DTA) models often excel within a benchmark yet fail under distribution shift across datasets, protein families, or structurally unconventional targets. This brittleness reflects a limitation of pair-based architectures: they memorize dataset-specific associations rather than learning transferable determinants of binding. We introduce Anchor Transfer Learning, which reformulates DTA prediction as a comparison problem. Instead of scoring a protein--drug pair in isolation, the model conditions each prediction on an anchor protein already known to bind a chemically similar drug, shifting the question from ``does protein P bind drug D?'' to ``how does P compare with a known binder of a related compound?'' At test time, anchors are retrieved from the training set by Tanimoto chemical similarity after excluding canonicalized chemical duplicates, requiring no privileged information about the evaluation data. We demonstrate that anchor transfer is architecture-agnostic by applying it to three distinct DTA architectures. On the Davis kinase benchmark under a cross-dataset protocol with verified zero canonical compound overlap: V2-650M achieves per-protein CI 0.642 and AUROC 0.669; AnchorDrugBAN improves DrugBAN from CI 0.483 to 0.645 (+0.162); and ConciseAnchor improves CoNCISE from CI 0.727 to 0.771 (+0.044) with AUROC rising from 0.806 to 0.887 under a unified Tanimoto cross-dataset protocol. On homolog-filtered Davis (<50% identity, 114 novel proteins), the improvement persists (CI +0.026, AUROC +0.066). On BindingDB, Tanimoto-retrieved anchors hurt overall performance due to protein family mismatch, but oracle anchors (dataset-internal, pK_i < 7, excluding self-predictions) reverse this: ConciseAnchor achieves CI 0.670 and AUROC 0.854 (vs CoNCISE's 0.617 and 0.782), winning across all anchor quartiles. These results establish anchor-based transfer as a general principle whose benefit is gated by anchor retrieval quality, applicable across protein encoders (ESM-2, Raygun, CNN) and drug representations (SMILES, molecular graphs, fingerprints).