In this paper we study Relational Reinforcement Learning in a multi-agent setting. There is growing evidence in the Reinforcement Learning research community that a relational representation of the state space has many bene ts over a propositional one. Complex tasks as planning or information retrieval on the web can be represented more naturally in relational form. Yet, this relational structure has not been exploited for multi-agent reinforcement learning tasks and has only been studied in a single agent context so far. This paper is a rst attempt in bridging the gap between Relation Reinforcement Learning (RRL) and Multi-agent Systems (MAS). More precisely, we will explore how a relational structure of the state space can be used in a Multi-Agent Reinforcement Learning context.