Modulation format identification (MFI) is a key technology in optical performance monitoring for the next-generation optical network, such as the intelligent cognitive optical network. An MFI scheme based on the Calinski–Harabasz index for a polarization-division multiplexing (PDM) optical fiber communication system is proposed. The numerical simulations were carried out on a 28 Gbaud PDM communication system. The results show that the required minimum optical signal-to-noise ratio values of each modulation format to achieve 100% identification accuracy are all equal to or lower than their corresponding 7% forward error correction thresholds, and the proposed scheme is robust to residual chromatic dispersion. Meanwhile, the proposed scheme was further verified by 20 Gbaud PDM-QPSK/16QAM/32QAM long-haul fiber transmission experiments. The results show that the scheme has a good reliability when fiber non-linear impairments exist. In addition, the complexity of the scheme is significantly lower than that of other clustering-based MFI schemes.