The deployment of beyond 5G and 6G networks introduces many new services with stringent Quality of Service (QoS) requirements. Recently machine learning has been shown to be a viable solution in proposing adaptable solutions. However, centralized machine learning based solutions still encounter hurdles in achieving real-time responsiveness due to their need of a global network view. In this paper, we explore a distributed approach aimed at optimizing network performance in real-time scenarios. By using Multi-Agent Systems (MAS), our method targets near-real-time end-to-end delay assurance across diverse network domains, without the need for prior traffic profile knowledge. Evaluated results highlight the effectiveness of our approach in reducing routing costs and ensuring desired end-to-end delay levels.