Consumer presence and nutrient availability can have contrasting and interactive effects on plant diversity. In a factorial experiment, we manipulated two levels of nutrient supply and the presence of two moderately specialized grazers in different combinations (no grazers, two species in monoculture, and both in combination). We tested how nutrients and grazers regulated the biomass of marine coastal epiphytes and the diversity of algal assemblages, based on the prediction that the effect of consumers on prey diversity depends on productivity and consumer specialization. Nutrient enrichment increased the epiphytic load, while monocultures of single grazer species partly prevented epiphyte growth. However, only the presence of two species with complementary feeding preferences effectively prevented epiphyte overgrowth. The epiphytes comprised micro- and macroalgal species, and the diversity of these algal assemblages differed, depending on grazer identity. For the microalgae, diversity was reduced by nutrient addition when grazer control was inefficient, but not when specialist microalgal grazers were present. Macroalgal diversity was reduced in ambient water with specialist macroalgal grazers compared to the treatment with inefficient ones. These results indicate that grazer composition and productivity are crucial in determining whether consumer pressure will have a positive or negative effect on algal diversity.