Presented at the 1st Workshop on AI for Urban Planning at AAAI 2025 as a non-archival workshop paper. Urban road systems require intricate planning to ensure safe and efficient transportation. Effective road systems help reduce traffic congestion, maximize throughput, and minimize collisions. Previous applications of machine learning algorithms to this topic have largely been focused on prediction, not optimization. In this work, we train and evaluate evolutionary and reinforcement learning models on this optimization problem by interfacing with the Simulation of Urban Mobility (SUMO) package. SUMO offers a framework to model and evaluate traffic dynamics, allowing users to configure parameters to explore the effect of various configurations on evaluation metrics. Models set values for traffic signal timings, speed limits, and designated lane access across a road network. They are evaluated on metrics for vehicle throughput, total waiting time, total travel time, speed variation, and crash frequency. We observe performance improvements compared to configurations that estimate the corresponding real-world networks, indicating evolutionary and reinforcement learning approaches might be well suited for this task, despite sparse application thus far.