This paper offers an interpretation of Wittgenstein’s remarks that discusses the meaning of being in practical agreement with a rule, arguing that Brandom misconstrues the idea undergirding Wittgenstein’s remarks in terms of the relation between the pragmatic and normative aspects of language. First, I discuss Brandom’s idea of normative pragmatism and Wittgenstein’s remarks on rule-following in the Philosophical Investigations. I argue that Brandom enforces the picture of implicit rules as a salient solution for the problem of infinite regress regarding explicit rules. Second, I compare both views and show that although Brandom takes his solution for a Wittgensteinian answer to the regress problem it is very likely that Wittgenstein’s understanding of rule-following rather suggests a different view. Moreover, I explain why Brandom thinks that he cannot accept this view and why he offers an interpretation-based account instead which he thinks is underlying the agreement between rule and practice. Third, I criticize Brandom’s account from a Wittgensteinian point of view arguing that what is underlying the agreement are so-called ‘bedrock-practices’ rather than mutual interpretations.