The use of Artificial Intelligence principles represents the next research challenge to support future network applications in the upcoming 6G era. In this work, we propose a novel approach: exploiting the principles of Reinforcement Learning (RL) and the availability of programmable switches to implement a new forwarding mechanism in the data plane of the 6G core network. More in detail, we define a Q-learning-based forwarding mechanism that acts at packet level and is able to select the minimum latency path at line rate. Our solution, referred to as Q-Learning-based Queue Length Routing in DAta Plane ((QL)2-RODAP), is fully decentralized and exploits in-band network telemetry to distribute network states among network nodes. We show that, either in random and real network topologies, our (QL)2-RODAP algorithm promptly reacts to sudden traffic bursts, and allows reducing the peak of queuing delays of about 65 − 85% with respect to other RL based approaches, thus cutting off the long tail of end-to-end latency that is critical for delay sensitive applications.