Frequency hopping signals have been widely used in military and civilian applications during the last few decades. It's of great value to locate emitters of frequency hopping signals, especially in military operations. Localization by time difference of arrival (TDOA) is a passive location method of good accuracy which utilizes the differences between the arrival times of signals received by multiple receivers at different sites. Many researches have been published on TDOA estimation algorithms for signals in passive location systems, but there have been few results in the literature about the TDOA estimation accuracy of frequency hopping signals. In this paper, a novel Cramer-Rao lower bound(CRLB) of TDOA estimation for frequency hopping signals is proposed and derived based on the given frequency hopping signal model. Theoretical analysis has revealed how frequency hopping signals' parameters, such as signal band, channel interval, hop duration and signal-to-noise ratio(SNR), affect the estimation performance. Simulation results have justified the theoretical derivation.