This dataset contains annotations of 31 herbarium specimens of Streptanhus tortuosus Kellogg for which we have we carefully and manually drew and annotated the contours of four reproductive organs: “bud”, “flower”, “immature fruit” and “mature fruit”. The dataset can be used to assess the ability of automated methods to count and detect precisely the shapes of these reproductive organs, with a view to conducting phenological studies. The annotations are formatted in accordance with the COCO data format, a usual format for object detection tasks in the field of Computer Vision. The annotations are divided into two files: train_21_full_masks.json contains the mask coordinates and labels of 21 herbarium sheets that can be used for training models test_10_full_masks.json contains the mask coordinates and labels of 10 other herbarium that can be used as a groundtruth file for evaluating the predictions, typically with the COCO evaluation scripts (https://github.com/cocodataset/cocoapi) Please refer to the following publication for a first assessment of this dataset with a Mask-RCNN approach: H. Goëau, A. Mora-Fallas, J. Champ, N. Love, S. Mazer, E. Mata-Montero, A. Joly, P. Bonnet. 2020. New fine-grained method for automated visual analysis of herbarium specimens: a case study for phenological data extraction. Applications in Plant Sciences