In the paper we deal with some computational aspects of the Generalized minimal residual method (GMRES) for solving systems of linear algebraic equations. The key question of the paper is the importance of the orthogonality of computed vectors and its influence on the rate of convergence, numerical stability and accuracy of different implementations of the method. Practical impact on the efficiency in the parallel computer environment is considered.