By a nilpotent map we mean an orientably regular map whose orientation preserving automorphism group is nilpotent. The nilpotent maps are concluded to the maps whose automorphism group is a $2$-group and a complete classification of nilpotent maps of (nilpotency) class $2$ is given by Malnič et al. in [European J. Combin. 33 (2012), 1974-1986]. It is proved by Conder et al. in [J. Algebraic Combin. 44 (2016), 863-874] that given the class, there are finitely many simple nilpotent maps. However, for the nilpotent maps with multiple edges and given class, since its automorphism group may be infinitely big, it is impossible to list it by a computer. Therefore, to classify the nilpotent maps with small class $c$ is necessary and interesting. In this paper, the nilpotent maps of class $4$ will be determined.