This paper presents a pavement segmentation and crack detection system from pavement images with complicated background information. The proposed method consists of three steps. In the first step, a Support Vector Machine, which shows a high degree of accuracy in classifying data, was employed to classify the image into two categories: a pavement group and a background group. In the second step, the crack was extracted by a fractal thresholding. Finally, a Radon Transform was applied to the crack image to classify the cracks into four different types. The experimental results show that the proposed system is robust and can effectively be used in pavement images with complicated background components such as trees, houses, etc.