We examine the economic and environmental impacts of improvements and disruptions in multimodal transport networks. Our quantitative spatial equilibrium model incorporates routing over multiple modes and congestion at intermodal terminals. We estimate a modal substitution elasticity with highway and rail data, and a terminal congestion elasticity with vessel-positioning data. Calibrated to the U.S. freight network, our model identifies key bottlenecks and quantifies $300-700 million in additional real GDP gains from intermodal terminal improvements. These gains are 2.5 times higher without congestion, and substitution away from roads yield additional environmental benefits. Losing rail network access, factoring in modal substitution and general equilibrium effects, is estimated to reduce real GDP by $230 billion.