The objective of this project is to automate the process of driving a car. The result of this project will surely reduce the number of hazards happening everyday. Our world is in progress and self driving car is on its way to reach consumer‟s door-step but the big question still lies that will people accept such a car which is fully automated and driverless. The idea is to create an autonomous Vehicle that uses only some sensors (collision detectors, temperature detectors etc.) and camera module to travel between destinations with minimal/no human intervention. The car will be using a trained Convolutional Neural Network (CNN) which would control the parameters that are required for smoothly driving a car. They are directly connected to the main steering mechanism and the output of the deep learning model will control the steering angle of the vehicle. Many algorithms like Lane Detection, Object Detection are used in tandem to provide the necessary functionalities in the car.