The destruction of man in the Shoah or Holocaust did not mean that Levinas argues in favor of turning away from the socio-historical reality to cultivate his own little garden. The deepest truth of subjectivity can be found in an alterity that calls for a socio-political responsibility. The political implications are rooted in different layers of Levinas’s thought. In his Talmudic comments, Levinas questions the reality of war as the truth of politics. But his explorations of subjectivity, ethical relationality and society allow to understand different political options such as contract theory (responsibility in the first person), liberation philosophy and human rights (responsibility in the second person) and the necessity of building a just society (ethics in the third person). Paradoxically, a just and equitable society ignores the uniqueness of the unique other. While organized responsibility is necessary, it introduces a new form of violence. In this article, we bring together the different layers in Levinas’s political vision and we explore its limits. A fundamental question is whether Levinas’s vision of politics is based on ethics or whether his ethics is a critique of politics.