Road defects like potholes have a major impact on road safety and comfort. Detecting these defects manually is a highly time consuming and expensive task. Previous approaches to detect road events automatically using acceleration sensors and gyro meters showed good results. However, these results could be significantly improved with additional usage of image analysis. A large, labelled image dataset is required for training and validation. This paper presents a method to generate this data in a fraction of the time needed as if the video is analysed manually. The method is based on a simple two step approach: at first, an unsupervised algorithm detects possible events based on the acceleration data and filters those video sequences with defects. Secondly, a human operator decides based on the short video sequences if the event was due to an existing road defect and labels the corresponding area in an image. A single operator can analyse 8 hours of video data in less than 20 minutes using this method, which corresponds to ~200 km labeled road data. The generated dataset is especially well fitted for sensor fusion tasks because each visual event is always mapped to a motion event.