This work presents a new interactive Content Based Image Retrieval (CBIR) scheme, termed Attribute Feedback (AF). Unlike traditional relevance feedback purely founded on low-level visual features, the Attribute Feedback system shapes users' information needs more precisely and quickly by collecting feedbacks on intermediate level semantic attributes. At each interactive iteration, AF first determines the most informative binary attributes for feedbacks, preferring the attributes that frequently (rarely) appear in current search results but are unlikely (likely) to be users' interest. The binary attribute feedbacks are then augmented by a new type of attributes, "affinity attributes", each of which is off-line learnt to describe the distance between user's envisioned image(s) and a retrieved image with respect to the corresponding affinity attribute. Based on the feedbacks on binary and affinity attributes, the images in corpus are further re-ranked towards better fitting the users' information needs. Extensive experiments on two real-world image datasets well demonstrate the superiority of the proposed scheme over other state-of-the-art relevance feedback based CBIR solutions.