A multi-objective adaptive dynamic programming method is proposed in this paper to solve the optimal control with multi-objective performance indices. The multi-objective performance indices are commonly used in engineering fields. In the paper, vector-valued performance indices are transformed into a scalar by vector's 2-norm. Multi-objective adaptive dynamic programming is derived from adaptive critic design and neural network. Heuristic Dynamic Programming (HDP) is a member of Adaptive Dynamic Programming. This method is implemented with Neural Network, so it does not require an accurate model of the plant. The dynamics of the system is learned from the history data, and from the data of real time control process. The proposed method updates the controller's parameters periodically, so the control law can be updated with the change of environment. It solves the problem step by step and optimizes the multi-objective performance indices in real time. The simulation shows the effectiveness of the method in the excitation control of synchronous generators.