ABSTRACTBoth arthropods and large grazing herbivores are important components and drivers of biodiversity in grassland ecosystems, but a synthesis of how arthropod diversity is affected by large herbivores has been largely missing. To fill this gap, we conducted a literature search, which yielded 141 studies on this topic of which 24 simultaneously investigated plant and arthropod diversity. Using the data from these 24 studies, we compared the responses of plant and arthropod diversity to an increase in grazing intensity. This quantitative assessment showed no overall significant effect of increasing grazing intensity on plant diversity, while arthropod diversity was generally negatively affected. To understand these negative effects, we explored the mechanisms by which large herbivores affect arthropod communities: direct effects, changes in vegetation structure, changes in plant community composition, changes in soil conditions, and cascading effects within the arthropod interaction web. We identify three main factors determining the effects of large herbivores on arthropod diversity: (i) unintentional predation and increased disturbance, (ii) decreases in total resource abundance for arthropods (biomass) and (iii) changes in plant diversity, vegetation structure and abiotic conditions. In general, heterogeneity in vegetation structure and abiotic conditions increases at intermediate grazing intensity, but declines at both low and high grazing intensity. We conclude that large herbivores can only increase arthropod diversity if they cause an increase in (a)biotic heterogeneity, and then only if this increase is large enough to compensate for the loss of total resource abundance and the increased mortality rate. This is expected to occur only at low herbivore densities or with spatio‐temporal variation in herbivore densities. As we demonstrate that arthropod diversity is often more negatively affected by grazing than plant diversity, we strongly recommend considering the specific requirements of arthropods when applying grazing management and to include arthropods in monitoring schemes. Conservation strategies aiming at maximizing heterogeneity, including regulation of herbivore densities (through human interventions or top‐down control), maintenance of different types of management in close proximity and rotational grazing regimes, are the most promising options to conserve arthropod diversity.