With the development of the Internet, new requirements are put forward on image compression effect, compression ratio and encoding and decoding time. The principle of fractal coding is novel. It breaks through the traditional entropy boundary theory. Especially its high compression ratio makes the fractal image coding become a hot research point. But this method has the encoding and decoding lopsided problem. Based on the basic principle of fractal image compression, t h is paper proposes a calculation method which is based on the main diagonal, namely only using the two diagonal data to do mean calculation. Thus not only reduce the data in operation, but also make the time be reduced to a certain degree. The experimental results show that, under the same quality of the image reconstruction, the algorithm makes the encoding and decoding time be greatly reduced.