AbstractThe propagation mechanism of zigzag dispersion curves in the Rayleigh wave exploration is theoretically analyzed. The propagation characteristics and interrelation of the guided waves are studied. It is suggested that the zigzag structure of Rayleigh wave is the result of the many modes generated by the low‐velocity layers or fractures in multilayered media. The inverse problems of the formation structure with the dispersion curves of the Rayleigh wave by the damping method of least squares and the genetic algorithm are analyzed. It is found that the damping method of least squares is not suitable to the inverse study of zigzag dispersion problem of Rayleigh wave while the genetic algorithm is a good method for this problem and can be used successfully.