SummaryThe approximate dynamic programming needs 2 prerequisites to be an effective optimal control method. Firstly, it must be assured to be stable and convergent before application. Secondly, the control system should mainly be a nonlinear multi‐input multi‐output form. Thus, this paper introduces a nonlinear multi‐input multi‐output approximate dynamic programming and proves that it is stable in Lyapunov sense, therefore it is convergent. Besides, the Lyapunov function design is also analyzed. These proofs are based on the Lyapunov stability theory in the form of the utility function of quadratic, square‐weighted sum, and absolute value. Thereafter, 3 typical control examples of nonlinear multi‐input multi‐output approximate dynamic programming are offered to show their applications and verify the proofs. The proof overcomes the complex derivation, and the results contain 3 practical and systematic bounded proofs. It is for the first time that the proof focuses on nonlinear multi‐input multi‐output approximate dynamic programming from the view of utility function. What is more, the results can also serve as an effective analysis and guide for the utility function design and the stability criterion of nonlinear multi‐input multi‐output approximate dynamic programming as well.