Identifying the plant species comprising a pasture, among other aspects, is crucial for assessing its nutritional value for grazing animals and facilitating its effective management. Traditionally, it requires labor-intensive visual inspection. Artificial Intelligence (AI) offers a solution for automatic classification, yet robust datasets for training such models in natural, uncontrolled environments are scarce. This data descriptor presents a dataset of 741 images collected in pasture lands in the Centre of Portugal using standard cameras at a height of 50 cm. A semi-automated annotation pipeline was employed, utilizing a Faster R-CNN model followed by manual verification and refinement. The dataset contains 1744 annotations across four categories: ‘Shrubs’, ‘Grasses’, ‘Legumes’, and ‘Others’. It includes diverse morphological variations and captures real-world challenges such as occlusion and lighting variability. This dataset serves as a benchmark for training object detection models in agricultural settings, facilitating the development of automated monitoring systems for precision agriculture. Such a mechanism could be incorporated into a mobile application, mounted on a drone, or embedded in an animal-worn device, enabling automated sampling and identification of the plant composition within a pasture.