The data set includes 589 T2-weighted images acquired from the same number of patients collected by seven studies, INDEX, the SmartTarget Biopsy Trial, PICTURE, TCIA Prostate3T, Promise12, TCIA ProstateDx (Diagnosis) and the Prostate MR Image Database. Further details are reported in the respective study references. These images were divided into seven subsets based on the acquiring institution. The cross-institution imaging protocols contain multiple scanners (two manufacturers with mixed 1.5 and 3T field strengths), varying field-of-view and anisotropic voxels, in-plane voxel dimensions ranging between 0.3 and 1.0 mm and out-of-plane spacing between 1.8 and 5.4 mm. For each image, eight anatomical structures of planning interest were labelled, including bladder, bone, obturator internus, transition zone, central gland, rectum, seminal vesicle and neurovascular bundle, respectively represented by class 1 to 8. All segmentations were manually annotated by eight biomedical imaging researchers, with experience ranging from 2 to 10 years in the annotation of medical image data, each annotating a mixed-institution subset using an institution-stratified sampling. Each annotation has been reviewed at least once. All images and labels could be found in data.zip while the indexing from image to trial and to institution are respectively provided in trial.txt and institution.txt. if you find this labelled data set useful for your research please consider to acknowledge the work: Li, Y., et al. "Prototypical few-shot segmentation for cross-institution male pelvic structures with spatial registration." arXiv preprint arXiv:2209.05160 (2022).