We present a phase retrieval method (PRM) for analyzing single-phase displacement interferometry measurements on rapidly changing velocity histories, including photon Doppler velocimetry (PDV). PRM identifies the peaks and valleys as well as zero-crossing points in a PDV time series, performs normalization and extracts point-by-point phase and thus velocity information. PRM does not require a wide time window as in sliding window Fourier transformation, and thus improves the effective temporal resolution. This method is implemented in analyzing PDV data obtained from gas gun experiments, and validated against simultaneous measurements with velocity interferometer system for any reflector.