We observe that the density of the Kummer distribution satisfies a certain differential equation, leading to a Stein characterization of this distribution and to a solution of the related Stein equation. A bound is derived for the solution and for its first and second derivatives. To provide a bound for the solution we partly use the same framework as in Gaunt 2017 [Stein, ESAIM: PS 21 (2017) 303–316] in the case of the generalized inverse Gaussian distribution, which we revisit by correcting a minor error. We also bound the first and second derivatives of the Stein equation in the latter case.