AbstractNumerical solving of real‐world problems typically consists of several stages. After a mathematical description of the problem and its proper reformulation and discretisation, the resulting linear algebraic problem has to be solved. We focus on this last stage, and specifically consider numerical stability of iterative methods in matrix computations. In iterative methods, rounding errors have two main effects: They can delay convergence and they can limit the maximal attainable accuracy. It is important to realize that numerical stability analysis is not about derivation of error bounds or estimates. Rather the goal is to find algorithms and their parts that are safe (numerically stable), and to identify algorithms and their parts that are not. Numerical stability analysis demonstrates this important idea, which also guides this contribution. In our survey we first recall the concept of backward stability and discuss its use in numerical stability analysis of iterative methods. Using the backward error approach we then examine the surprising fact that the accuracy of a (final) computed result may be much higher than the accuracy of intermediate computed quantities. We present some examples of rounding error analysis that are fundamental to justify numerically computed results. Our points are illustrated on the Lanczos method, the conjugate gradient (CG) method and the generalised minimal residual (GMRES) method.