SummaryThe Neumann problem of linear elasticity is singular with a kernel formed by the rigid motions of the body. There are several tricks that are commonly used to obtain a nonsingular linear system. However, they often cause reduced accuracy or lead to poor convergence of the iterative solvers. In this paper, different well‐posed formulations of the problem are studied through discretization by the finite element method, and preconditioning strategies based on operator preconditioning are discussed. For each formulation, we derive preconditioners that are independent of the discretization parameter. Preconditioners that are robust with respect to the first Lamé constant are constructed for the pure displacement formulations, whereas a preconditioner that is robust in both Lamé constants is constructed for the mixed formulation. It is shown that, for convergence in the first Sobolev norm, it is crucial to respect the orthogonality constraint derived from the continuous problem. On the basis of this observation, a modification to the conjugate gradient method is proposed, which achieves optimal error convergence of the computed solution.