In this paper, we investigate the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent's local objective function, which cannot be known by other agents. The communication network is assumed to switch over a set of weight-balanced directed graphs with a Markovian property.We propose a consensus sub-gradient algorithm with two time-scale step-sizes to handle the uncertainty due to the Markovian switching topologies and the absence of global gradient information. With a proper selection of step-sizes, we prove the almost sure convergence of all agents' local estimates to the same optimal solution when the union graph of the Markovian network' states is strongly connected and the Markovian network is irreducible. Simulations are given for illustration of the results.

10 pages, 2 figures