A crucial ingredient in social interaction models is the structure of peer groups, which link individuals with similar characteristics. We propose and study a dynamic binary choice model with social interactions in which heterogeneity of peer group effects is modeled introducing diversity in individual characteristics and linking pairwise influences to a social distance between individuals. Our framework allows for mimetic as well as anti-mimetic interactions and a heterogeneous structure of peer groups across individuals. Dynamic equilibria are studied in the limit when the number of agents is large. We show that the model exhibits multiple equilibria resulting from conflicts between various group pressures the individuals are subjected to. We study in particular the correlation in the population at equilibrium between the characteristics of the agents and their decisions: this quantity has an interesting empirical interpretation and solves a simple analytical equation when the number of agents is large. Finally we discuss the empirical content of the model and present a consistent estimator for the parameter describing which is consistent for any typical population regardless of the structure of individual characteristics.