Audio forgery is a critical part of digital forgery. There is a large variety of audio forgery forms, and copy-move is one of the most common but effective one. Merely one word's copy-move can turn the whole sentence into distinct meaning. So the way to detect this form of forgery precisely and effectively is especially useful and significant in the field of digital forensics. In this paper, we propose a fast method to detect audio copy-move forgery. At first, the audio is segmented into syllables. Then discrete Fourier transform (DFT) on each audio segment is applied for the sake of extracting the feature. In addition, every audio segment is sorted based on the features. Finally, unlike other mainstream algorithms which compare the similarity between every two segments of the audio file, we just compare one audio segment with some adjacent ones in the sorted list of audio segments, which can reduce the time complexity from O(n2) to O(nlogn) and cut down the time consumption of detection consequently. Experimental results demonstrate that our methods is faster and more effective than other state of the art ones.