A well known conjecture in complex geometry states that a compact Hermitian manifold with constant holomorphic sectional curvature must be Kähler if the constant is non-zero and must be Chern flat if the constant is zero. The conjecture is confirmed in complex dimension $2$, by the work of Balas-Gauduchon in 1985 (when the constant is zero or negative) and by Apostolov-Davidov-Muskarov in 1996 (when the constant is positive). For higher dimensions, the conjecture is still largely unknown. In this article, we restrict ourselves to the class of complex nilmanifolds and confirm the conjecture in that case.