Dataset of greenhouse tomatoes for object detection in Pascal VOC. This dataset was collected on two different days (August 6 and 8, 2020) at a greenhouse in Barroselas, Viana do Castelo, Portugal. Mobile robot AgRob v16, controlled by a human operator, was guided through the greenhouse inter-rows and captured RGB images of the tomato plants using a ZED camera, recording them as a video in a single ROSBag file. The video was converted into images by sampling a frame every 3 seconds to reduce the correlation between images but ensuring an overlapping ratio of about 60%. The images collected on the two days were merged, resulting in a dataset of 449 images with a resolution of 1280x720 px each. The images captured on August 8, 2020 were already publicly available at INESC TEC Research Data Repository (see: https://rdm.inesctec.pt/dataset/ii-2021-001). Manually labelled using the CVAT annotation tool, considering 4 ripeness classes: Unriped; Breaking Stage; Reddish; Riped; Framed within the activities of the ROBOCARE project, P2020 developed by INESC TEC (https://www.inesctec.pt/pt/projetos/robocare)