The complexity of electrochemical interfaces has led to the development of several approximate density functional theory (DFT)-based schemes to study reaction thermodynamics and kinetics as a function of electrode potential. While fixed electrode potential conditions can be simulated with grand canonical ensemble DFT (GCE-DFT), various electrostatic corrections on canonical, constant charge DFT are often applied instead. In this work, we present a systematic derivation and analysis of the different electrostatic corrections on canonical DFT to understand their physical validity, implicit assumptions, and scope of applicability. Our work highlights the need to carefully address the suitability of a given model for the problem under study, especially if physical or chemical insight in addition to reaction energetics is sought. In particular, we analytically show that the different corrections cannot differentiate between electrostatic interactions and covalent or charge-transfer interactions. By numerically testing different models for CO2 adsorption on a single-atom catalyst as a function of the electrode potential, we further show that computed capacitances, dipole moments, and the obtained physical insight depend sensitively on the chosen approximation. These features limit the scope, generality, and physical insight of these corrective schemes despite their proven practicality for specific systems and energetics. Finally, we suggest guidelines for choosing different electrostatic corrections and propose the use of conceptual DFT to develop more general approximations for electrochemical interfaces and reactions using canonical DFT.