Interfacial fields within organic photovoltaics influence the movement of free charge carriers, including exciton dissociation and recombination. Open circuit voltage (Voc) can also be dependent on the interfacial fields, in the event that they modulate the energy gap between donor HOMO and acceptor LUMO. A rise in the vacuum level of the acceptor will increase the gap and the Voc, which can be beneficial for device efficiency. Here, we measure the interfacial potential differences at donor-acceptor junctions using Scanning Kelvin Probe Microscopy, and quantify how much of the potential difference originates from physical contact between the donor and acceptor. We see a statistically significant and pervasive negative polarity on the phenyl-C61 butyric acid methyl ester (PCBM) side of PCBM/donor junctions, which should also be present at the complex interfaces in bulk heterojunctions. This potential difference may originate from molecular dipoles, interfacial interactions with donor materials, and/or equilibrium charge transfer due to the higher work function and electron affinity of PCBM. We show that the contact between PCBM and poly(3-hexylthiophene) doubles the interfacial potential difference, a statistically significant difference. Control experiments determined that this potential difference was not due to charges trapped in the underlying substrate. The direction of the observed potential difference would lead to increased Voc, but would also pose a barrier to electrons being injected into the PCBM and make recombination more favorable. Our method may allow unique information to be obtained in new donor-acceptor junctions.