Atomic magnetometers (AMs) offer many advantages over superconducting quantum interference devices due to, among other things, having comparable sensitivity while not requiring cryogenics. One of the major limitations of AMs is the challenge of configuring them as gradiometers. We report the development of a spin-exchange relaxation free vector atomic magnetic gradiometer with a sensitivity of 3 fT cm−1 Hz−1/2 and common mode rejection ratio >150 in the band from DC to 100 Hz. We introduce a background suppression figure of merit for characterizing the performance of gradiometers. It allows for optimally setting the measurement baseline and for quickly assessing the advantage, if any, of performing a measurement in a gradiometric mode. As an application, we consider the problem of fetal magnetocardiography (fMCG) detection in the presence of a large background maternal MCG signal.