Automatic image annotation is a challenging field with a far reaching effect. As the world moves towards becoming more and more dependent on digital technologies every day, use of machine to automatically annotate images can be proved as demanding in many fields of image processing. Automatic Image Annotation reduces the gap between low level image features and high level image semantics. Utilization of Speeded Up Robust Features (SURF) in automatic image annotation is very appealing due to the fact that SURF is scale and rotation invariant detector and descriptor and is much faster than any other schemes. Unlike other methods SURF features use the entire image instead of segmented blocks of image. That is why annotation of images by using SURF can be considered as more accurate. In this paper, a SVM based image annotation approach is proposed that uses SURF features of image for annotation purpose. The experiments suggest that the method proposed is much more efficient than other methods. General Terms Pattern Recognition, Image Annotation.