We consider the problem of temporal fair scheduling of queued data transmission in wireless networks. Taking fairness constraints and the memory property of channels into consideration, we formulate the transmission scheduling problem as a discounted reward Markov decision process (MDP) with temporal fairness constraints. We derive and prove an explicit dynamic programming equation for the above constrained MDP, and give an optimal scheduling policy based on that equation. Furthermore, we develop an efficient approximation method-temporal fair rollout-to reduce the computational cost. The simulation results show that the scheme achieves significant performance improvement for both throughput maximization and delay minimization problems compared with other existing schemes.