Usurpation is a common term used by Western historians to describe the illegitimate change of political supremacy inthe context of medieval Muslim states. The taking over of any state without authority is considered an illegal occupationof a legitimate state and its leader is considered illegitimate or usurper. This paper attempts to shed some lights on thenotion of political legitimacy and authority in Islamic tradition and its application in the context of medieval Syria,particularly during the Zengid dynasty. This period experienced the coming of the second crusade to the East with therevitalization of the spirit of jihad was on its way among the Syrian Muslims. This paper argues that the Zengid dynastywas trying to uphold the institution of the caliphate through recognizing the spiritual leadership of the ‘Abbasid caliphof Baghdad as well as acquiring political legitimacy to administer their subject on behalf of the caliph and the Seljuqsultan. Through adherence to the Sunni tradition of political legitimacy, Zengi (d. 541/1146) and Nur al-Din (d.569/1174) succeeded in promoting Sunnism by means of Muslim unity and jihad enterprise. As a result, after theannexation of Egypt from the Fatimid caliphate in 565/1171, Muslims in Syria and Egypt were unified under the bannerof Sunnism with Nur al-Din as their new legitimate ruler