Relations and combinations between holographic interferometry and speckle correlation in contouring by phase‐shifting digital holography are discussed. Three‐dimensional distributions of correlations of the complex amplitudes and intensities before and after the laser wavelength shift are calculated in numerical simulations where a rough surface is modeled with random numbers. Fringe localization related to speckle displacement as well as speckle suppression in phase analysis are demonstrated for general surface shape and recording conditions.