Advanced Driver Assistance Systems benefit from a full 3D reconstruction of the environment in real-time, often obtained via stereo vision. Semi-Global Matching (SGM) is a popular stereo algorithm for solving this task which is already in use for production vehicles. Despite this progess, one key challenge remains: stereo vision during adverse weather conditions such as rain, snow and low-lighting. Current methods generate many disparity outliers and false positives on a segmentation level under such conditions. These shortcomings are alleviated by integrating prior scene knowledge. We formulate a scene prior that exploits knowledge of a representative traffic scene, which we apply to SGM and Graph Cut based disparity estimation. The prior is learned from traffic scene statistics extracted during good weather. Using this prior, the object detection rate is maintained on a driver assistance database of 3000 frames including bad weather while reducing the false positive rate significantly. Similar results are obtained for the KITTI dataset, maintaining excellent performance in good weather conditions. We also show that this scene prior is easy and efficient to implement both on CPU platforms and on reconfigurable hardware platforms. The concept can be extended to other application areas such as indoor robotics, when prior information of the disparity distribution is gathered.