Abstract Deep learning can revolutionise high-throughput image-based phenotyping by automating the measurement of complex traits, a task that is often labour-intensive, time-consuming, and prone to human error. However, its precision and adaptability in accurately phenotyping organ-level traits, such as fruit morphology, remain to be fully evaluated. Establishing the links between phenotypic and genotypic variation is essential for uncovering the genetic basis of traits and can also provide an orthologous test of pipeline effectiveness. In this study, we assess the efficacy of deep learning for measuring variation in fruit morphology in Arabidopsis using images from a multiparent advanced generation intercross (MAGIC) mapping family. We trained an instance segmentation model and developed a pipeline to phenotype Arabidopsis fruit morphology, based on the model outputs. Our model achieved strong performance with an average precision of 88.0% for detection and 55.9% for segmentation. Quantitative trait locus analysis of the derived phenotypic metrics of the MAGIC population identified significant loci associated with fruit morphology. This analysis, based on automated phenotyping of 332,194 individual fruits, underscores the capability of deep learning as a robust tool for phenotyping large populations. Our pipeline for quantifying pod morphological traits is scalable and provides high-quality phenotype data, facilitating genetic analysis and gene discovery, as well as advancing crop breeding research.