This research focused on understanding the effects of structural heterogeneity within tree crowns on the airborne retrieval of solar-induced chlorophyll fluorescence (SIF) and the Crop Water Stress Index (CWSI). We explored the SIF and CWSI variability observed within crowns of trees subjected to different water stress regimes and its effect on the relationships with leaf physiological measurements. High-resolution (20 cm) hyperspectral imagery was acquired to assess fluorescence retrieval from sunlit portions of the tree crowns using the Fraunhofer line depth method, and from entire crowns using automatic object-based tree crown detection methods. We also measured the canopy temperature distribution within tree crowns using segmentation algorithms based on temperature percentiles applied to high-resolution (25 cm) thermal imagery. The study was conducted in an almond orchard cultivated under three watering regimes in Cordoba, in southern Spain. Three airborne campaigns took place during the summer of 2015 using high-resolution hyperspectral and thermal cameras on board a manned aircraft. Relationships between SIF and the assimilation rate improved significantly when the sunlit tree crown pixels extracted through segmentation were used for all flight dates. By contrast, the SIF signal extracted from the entire tree crowns was highly degraded due to the canopy heterogeneity observed within tree crowns. The quartile crown segmentations applied to the thermal images showed that the CWSI values obtained were within the theoretically expected CWSI range only when the pixels were extracted from the 50th percentile class. However, the CWSI values were biased in the upper quartile (Q75) for all watering regimes due to the soil background effects on the calculated mean crown temperature. The relationship between the CWSI and Gs was heavily affected by the crown segmentation levels applied and improved remarkably when the CWSI values were calculated from the middle quartile crown segmentation (Q50), corresponding to the coldest and purest vegetation pixels (r2 = 0.78 in pure vegetation pixels vs. r2 = 0.52 with the warmer pixels included in the upper quartile). This study highlights the importance of using high-resolution hyperspectral and thermal imagery for pure-object segmentation extractions from tree crowns in the context of precision agriculture and water stress detection.