In this paper, we show how to solve large sparse linear systems in a grid environment using the Java language and the MPJ library for communication. We describe a parallel version of the GMRES method which takes into account the sparsity of the matrix for message exchanges among processors. Two implementations are compared: one in Java using MPJ and one in C using MPI. The performance of both codes is also compared with that of the PETSc library. Experiments have been carried out using the GRID'5000 platform, on the one hand, on a local cluster, and, on the other hand, on clusters located in distant geographical sites. It is noticeable that the performance of our solver in Java is comparable to the same solver written in C and also to the PETSc library. Our solver in Java allowed us to solve sparse systems of size up to 2 billions with two geographically distant sites.