Over the years, autonomous driving has attracted more and more interest from academia and industry. The field involves a lot of technologies, including SLAM, object detection and end to end driving decision. In this paper, our goal is to make driving decisions based on visual models, which is a hot topic both in computer vision and autonomous driving. Our work consists of three parts. First, we propose an 3DCNN end-to-end driving model that combines Residual Neural Networks (ResNet) with Convolutional Long Short-Term Memory (Conv-LSTM), which includes spatiotemporal features at multiple scales. Second, we built dynamic and static traffic scenarios in the simulation platform and collected driver's driving data. Third, in order to prove the validity of the proposed model, it is compared with PilotNet on the public Udacity dataset. Furthermore, in order to improve the generalization of the model, we validate proposed model on our simulation dataset, the results show that our model can accurately predict the steering angle on different datasets.