It is appealing to stabilize dark matter by the same discrete symmetry that is used to explain the structure of quark and lepton mass matrices. However, to generate the observed fermion mixing patterns, any flavor symmetry must necessarily be broken, rendering dark matter unstable. We study singlet, doublet and triplet SU(2) multiplets of both scalar and fermion dark matter candidates and enumerate the conditions under which no d < 6 dark matter decay operators are generated even in the case if the flavor symmetry is broken to nothing. We show that the VEVs of flavon scalars transforming as higher multiplets (e.g. triplets) of the flavor group must be at the electroweak scale. The most economical way for that is to use SM Higgs boson(s) as flavons. Such models can be tested by the LHC experiments. This scenario requires the existence of additional Froggatt-Nielsen scalars that generate hierarchies in Yukawa couplings. We study the conditions under which large and small flavor breaking parameters can coexist without destabilizing the dark matter.

8 pages, no figures