In this paper, we present a new adaptive dynamic programming (ADP) scheme to solve the optimal control problem of multi-player systems with unknown dynamics from the perspective of nonzero-sum (NZS) games. In the presented scheme, a new iterative equation is given. On the basis of the given iterative equation, the control policy and corresponding value function for each player can be learned by using the state and input data, which does not need to identify the system dynamics. To overcome the difficulty of unknown system dynamics, neural network (NN)-based function approximation techniques are employed in the implementation. Based on the given iterative equation and NN-based function approximation techniques, a new non-model-based ADP algorithm is developed. The convergence of the developed non-model-based ADP algorithm is rigorously analyzed and proved. Finally, two numerical simulation examples are provided to demonstrate the performance of the developed non-model-based ADP algorithm.