This studies the infinite horizon optimal control problem for a class of continuous‐time systems subjected to multiplicative noises and Markovian jumps by using a data‐driven policy iteration algorithm. The optimal control problem is equivalent to solve a stochastic coupled algebraic Riccatic equation (CARE). An off‐line iteration algorithm is first established to converge the solutions of the stochastic CARE, which is generalised from an implicit iterative algorithm. By applying subsystems transformation (ST) technique, the off‐line iterative algorithm is decoupled into N parallel Kleinman's iterative equations. To learn the solution of the stochastic CARE from N decomposed linear subsystems data, an ST‐based data‐driven policy iteration algorithm is proposed and the convergence is proved. Finally, a numerical example is given to illustrate the effectiveness and applicability of the proposed two iterative algorithms.