Abstract A new set of relative orbit elements is strictly defined through spherical geometry. The exact transformation equations between the new relative orbit elements and classical-orbital elements are derived. A new relative motion model with no singularity problem is derived based on the relative orbit elements, which are suitable for both elliptical and circular reference orbits. The in-plane and out-of-plane relative motion can be completely decoupled based on the new model. The inverse transformation of state transfer matrix is obtained to analyze perturbation effects and control strategy. The geometric characteristics of relative motion can be easily described using the relative eccentricity/inclination vector method. The proposed method and conclusions are validated by simulation through some typical examples. This paper improves the basic theory of relative orbit elements and unifies the expressions of the elliptical and near-circular close relative motion.