Here we present SERS spectra of several l-phenylalanine (Phe) phosphonodipeptides, i.e., l-Phe-l-Ala-PO3H2 (MD1), l-Phe-l-Val-PO3H2 (MD2), l-Phe-β-Ala-CH(OH)-PO3H2 (MD3), l-Phe-l-Ala-CH(OH)-PO3H2 (MD4), l-Ala-(3,4-dimethoxy)-l-Phe-PO3H2 (MD5), and l-Ala-(3,4-dimethoxy)-(des-CH2)-l-Phe-PO3H2 (MD6), immobilized on electrochemically roughened silver electrodes. These spectra are analyzed by theoretical calculations using density functional theory (DFT) at the B3LYP level with 6-31++G∗∗ basis set. In addition, these spectra are compared with SERS spectra of these species adsorbed on a colloidal silver surface. We showed that on the macroscopic silver substrate, the Phe aromatic ring of MD3 and MD4 is oriented vertically, while for MD1 it almost “stands up” on this surface. In the other three cases, the Phe ring adopts a tilted orientation in regard to the substrate. We also find that the phosphonate (–PO32-), methyl/methane, or dimethoxy groups of MD1, MD2, MD3, MD5, and MD6 are involved in the interaction of these phosphonodipeptides with the electrochemically roughened surface. This phenomenon is clearly seen for –CH2–/–CH3/–OCH3 moieties as well as for the PO32- group that adsorbs on the macroscopic silver substrates mainly via the PO fragment. We also showed that MD4 binds to the macroscopic silver substrate through the hydroxyl, amine, and phosphonate groups, while the methylene/methane moieties are remote from this surface. We found that studied phosphonodipeptides often adsorb differently on the macroscopic silver substrate and on the colloidal silver nanoparticles. For example, MD1 adopts an almost vertical orientation on the electrochemically roughened silver substrate and is tilted or close to flat on the silver nanoparticles.