This paper was intended to explore the mutual influences between electric vehicle (EV) charging and charging facility planning, to establish a two-stage model for optimizing the EVs’ charging and charging piles’ selection. In the first stage, the distribution pattern of the demands for EV charging, and various EVs were effectively grouped, in order to reduce the amount of computation for solving the second stage model. The goal of the second stage was to minimize the annual investment and electricity purchasing costs on the charging piles, and the coordinated optimization was carried out for EV charging and charging pile selection. The CPLEX and IP_SOLVE packages were used in MATLAB (R2014a/64 bits) to solve the established optimization model. The simulation results showed that, compared with the scheme for selecting the charging pile under the typical charging pattern (TCP), the total cost of the charging pile could be reduced by 6.32% with a scheme under the optimized charging pattern (OCP), thereby promoting the coordinated development of both the EVs and charging facilities.