Abstract Why many herbivorous insects are host plant specialists, with non‐negligible exceptions, is a conundrum of evolutionary biology, especially because the host plants are not necessarily optimal larval diets. Here, I present a novel model of host plant preference evolution of two insect species. Because habitat preference evolution is contingent upon demographic dynamics, I integrate the evolutionary framework with the modern coexistence theory. The results show that the two insect species can evolve into a habitat specialist and generalist, when they experience both negative and positive frequency‐dependent community dynamics. This happens because the joint action of positive and negative frequency dependence creates multiple (up to nine) eco‐evolutionary equilibria. Furthermore, initial condition dependence due to positive frequency dependence allows specialization to poor habitats. Thus, evolved habitat preferences do not necessarily correlate with the performances. The model provides explanations for counterintuitive empirical patterns and mechanistic interpretations for phenomenological models of niche breadth evolution.