The calculation of the solution or Nash equilibrium of a differential game with N players involves solving N coupled HJB equations simultaneously. It becomes more complicated as N becomes larger due to increased interactions and coupling among the players. Consequently, mean field game (MFG) has been proposed by Lasry and Lions to reformulate a differential game problem. In an MFG, the aggregate effect of all the players is considered rather than the individual effect of each player. MFGs have been applied in many applications in economics and engineering in which the number of players is large and when the players are indistinguishable yet can have heterogeneous states.