We propose an equilibrium interaction model of occupational segregation and labor market inequality between two social groups, generated exclusively through the documented tendency to refer informal job seekers of identical "social color". The expected social color homophily in job referrals strategically induces distinct career choices for individuals from different social groups, which further translates into stable partial occupational segregation equilibria with sustained wage and employment inequality -- in line with observed patterns of racial or gender labor market disparities. Supporting the qualitative analysis with a calibration and simulation exercise, we furthermore show that both first and second best utilitarian social optima entail segregation, any integration policy requiring explicit distributional concerns. Our framework highlights that the mere social interaction through homophilous contact networks can be a pivotal channel for the propagation and persistence of gender and racial labor market gaps, complementary to long studied mechanisms such as taste or statistical discrimination.