Ecosystems' responses to changing environmental conditions can be modulated by spatial self-organization. A prominent example of this can be found in drylands, where formation of vegetation patterns attenuates the magnitude of degradation events in response to decreasing rainfall. In model studies, the pattern wavelength responds to changing conditions, which is reflected by a rather gradual decline in biomass in response to decreasing rainfall. Although these models are spatially explicit, they have adopted a mean-field approach to grazing. By taking into account spatial variability when modeling grazing, we find that (over)grazing can lead to a dramatic shift in biomass, so that degradation occurs at rainfall rates that would otherwise still maintain a relatively productive ecosystem. Moreover, grazing increases the resilience of degraded ecosystem states. Consequently, restoration of degraded ecosystems could benefit from the introduction of temporary small-scale exclosures to escape from the basin of attraction of degraded states.