For the large-scale distributed graph mining, the graph is distributed over a cluster of nodes, thus performing computations on the distributed graph is expensive when large amount of data have to be moved between different computers. A good partitioning of distributed graph is needed to reduce the communication between computers and scale a system up. Existing graph partitioning algorithms incur high computation and communication cost when applied on large distributed graphs. A efficient and scalable partitioning algorithm is crucial for large-scale distributed graph mining.In this paper, we propose a novel parallel multi-level stepwise partitioning algorithm. The algorithm first efficiently aggregates the large graph into a small weighted graph, and then makes a balance partitioning on the weighted graph based on a stepwise minimizing RatioCut Algorithm. The experimental results show that our algorithm generally outperforms the existing algorithms and has a high efficiency and scalability for large-scale graph partitioning. Using our partitioning method, we are able to greatly speed up PageRank computation.