Vision-based grape yield estimation provides a cost-effective solution for intelligent orchards. However, unstructured background, occlusion and dense berries make it challenging for grape yield estimation. We propose an efficient two-stage pipeline TSGYE: precise detection of grape clusters and efficient counting of grape berries. Firstly, high-precision grape clusters are detected using object detectors, such as Mask R-CNN, YOLOv2/v3/v4. Secondly, based on the detected clusters, berry counted through image processing technology. Experimental results show that TSGYE with YOLOv4 achieves 96.96% mAP@0.5 score on WGISD, better than the state-of-the-art detectors. Besides we manually annotate all test images of WGISD and make it public with a grape berry counting benchmark. Our work is a milestone in grape yield estimation for two reasons: we propose an efficient two-stage grape yield estimation pipeline TSGYE; we offer a public test set in grape berry counting for the first time.