This paper provides the first convergence proof for fuzzy reinforcement learning. We extend the work of Konda and Tsitsiklis (2000), who presented a convergent actor-critic algorithm for a general parameterized actor. In our work we prove that a fuzzy rule base actor satisfies the necessary conditions that guarantee the convergence of its parameters to a local optimum. Our fuzzy rule base uses the Takagi-Sugeno-Kang rules, Gaussian membership functions and product inference.