We consider spin chain models with local Hamiltonians that display weak ergodicity breaking. In these models, the majority of the eigenstates are thermal, but there is a distinguished subspace of the Hilbert space in which ergodicity is broken. We achieve such a weak breaking by embedding selected integrable models into larger Hilbert spaces of otherwise chaotic models. The integrable subspaces do not have a tensor product structure with respect to any spatial bipartition, therefore our constructions differ from certain trivial embeddings. We consider multiple mechanisms for such an embedding, and we also review previous examples in the literature. Curiously, all our examples can be seen as perturbations of models with Hilbert space fragmentation, such that the perturbed models are not fragmented anymore.