Coupling of control and perception is an especially difficult problem. This thesis investigates this problem in the context of aggressive off-road driving. By jointly developing a robust 1:5 scale platform and leveraging state of the art sampling based model predictive control, the problem of aggressive driving on a closed dirt track using only monocular cam- era images is addressed. It is shown that a convolutional neural network can directly learn a mapping from input images to top-down cost map. This cost map can be used by a model predictive control algorithm to drive aggressively and repeatably at the limits of grip. Further, the ability to learn an end-to-end trained attentional neural network gaze strategy is developed that allows both high performance and better generalization at our task of high speed driving. This gaze model allows us to utilize simulation data to generalize from our smaller oval track to a much more complex track setting. This gaze model is compared with that of human drivers performing the same task. Using these methods, repeatable, aggressive driving at the limits of handling using monocular camera images is shown on a physical robot. ; Ph.D.