As wireless networks become increasingly important in modern society, their application scenarios are becoming more diverse and complex. However, the heterogeneity of nodes and transmission conditions presents significant challenges to existing wireless strategies and traditional centralized AI methods, making it difficult to meet user demands for network throughput. This paper proposes a distributed architecture based on multi-agent reinforcement learning combined with deep reinforcement learning. Agents are deployed on individual transmission nodes, enabling distributed observation and autonomous decision-making, while the access point provides feedback derived from the network performance resulting from their individual decisions. By experimentally comparing centralized and distributed architectures in multi-rate environments, this paper analyzes trade-offs in scalability and network performance. Additional experiments conducted under dynamic network conditions with node mobility and static scenarios involving a larger number of coexisting nodes further validate the system’s robustness and adaptability. The analysis of training loss trends shows that although the distributed architecture incurs a higher training cost, it achieves improved throughput. In particular, the distributed method outperforms the centralized method by nearly 30% when the number of nodes is relatively small, and maintains a 5–10% performance advantage as the network continues to scale.