Many popular attempts to explain the observed patterns of fermion masses involve a flavon field. Such weakly coupled scalar fields tend to dominate the energy density of the universe before they decay. If the flavon decay happens close to the electroweak transition, the right-handed electrons stay out of equilibrium until the sphalerons shut off. We show that an asymmetry in the right-handed charged leptons produced in the decay of a flavon can explain the baryon asymmetry of the universe.