Fully automated detection and localisation of fruit in orchards are key components in creating automated robotic harvesting systems. During recent years a lot of research on this topic has been performed, either using basic computer vision techniques, like colour based segmentation, or by resorting to other sensors, like LWIR, hyperspectral or 3D. Recent advances in computer vision present a broad range of advanced object detection techniques that could improve the quality of fruit detection from RGB images drastically. We suggest to use an object categorisation framework based on boosted cascades of weak classifiers to implement a fully automated semi-supervised fruit detector and demonstrate it on both strawberries and apples. Compared to existing techniques we improved fruit detection, mainly in the case of fruit clusters, using a supervised machine learning instead of hand crafting image filters specific to the application. Moreover we integrate application specific colour information to ensure a more stable output of our fully automated detection algorithm. Finally we make suggestions for efficient fruit cluster separation. The developed technique is validated on both strawberries and apples and is proven to have large benefits in the field of automated harvest and crop estimation.