Dilepton spectra are a classic probe to study ultra-relativistic heavy ion collisions. At √ sNN = 200 GeV, the dimuon continuum is dominated by correlated pairs from semi-leptonic decays of charm and bottom hadrons and the Drell-Yan process. Measuring the azimuthal correlations of heavy flavor decay lepton pairs can help constrain the relative contributions from different heavy flavor production mechanisms in p+p collisions, and may provide further insight on cold nuclear matter effects in p+A collisions. Measuring the Drell-Yan cross-section in p+A collisions can provide constraints to nuclear parton distribution functions, and further our understanding in initial state effects. In this dissertation, measurements of muon pairs from charm, bottom, and Drell-Yan, using data collected by the Relativistic Heavy Ion Collider (RHIC) and the Pioneering High Energy Nuclear Interaction eXperiment (PHENIX), in p+p and p+Au collisions at √ sNN = 200 GeV are presented.