Use of herbicides is rising globally to maximize crop yield and profitability. Herbicides negatively impact environmental health and biosphere. To lessen its negative effects, herbicides have to be applied judiciously on crops. Precision agriculture practices suggest adoption of site specific weed management techniques by exploiting patchy nature of weed distribution in the fields which requires accurate weed mapping. Despite recent technical advancement and growing awareness about environment protection, site specific weed management has not got traction in farmer community. In this thesis, endeavours are made to develop relatively simple site specific weed control method using weed density based variable rate herbicide application. Soil, Water and Topography (SWAT) maps are being used by farmers for variable rate seeding and fertilizer in prairie lands of Canada. In this work, we investigate relationship between weeds and SWAT zones and present a new method for variable rate herbicide application which combines deep learning and SWAT maps. Average weed densities are estimated in each SWAT zone through deep learning based semantic segmentation in order to help agronomist develop variable rate herbicide prescription. The study simplifies the weed detection system with the objective to enhance savings of herbicide quantities less costs involved in site specific weed control. Manual labeling bottleneck in semantic segmentation is addressed by labeling only weed pixels. Consequently, trained semantic models zeros out crop pixel along with background pixels. The developed model has the advantage to detect new types of weeds. Binary classification of images based on weeds is also studied in this thesis to compare deep learning models. By investigating SWAT zones and weed density relationship, it is found that the zones with higher salinity, organic matter and water content contain higher density of weeds while the driest zones like eroded hill tops have few or no weeds at all. The crop specific semantic segmentation models have shown MIOU values greater than 80% and FWIOU values more than 97%. The trained models also show robustness in detecting unseen weeds. For binary classification problem of detecting weeds in Canola field, VGG19 has shown 100% accuracy compared to other deep learning architectures.