The female mammary epithelium undergoes reorganization during development, pregnancy, and menopause, linking to higher risks of developing breast cancer. To characterize these periods of complex remodeling, here we report integrated 50K mouse and 24K human mammary epithelial cell atlas obtained by single-cell RNA sequencing, which covers most lifetime stages. Our results indicate a putative trajectory that originates from embryonic mammary stem cells which differentiates into three epithelial lineages (basal, luminal hormone-sensing, and luminal alveolar), presumably arising from unipotent progenitors in postnatal glands. The identified lineage-specific genes inferred cells of origin of breast cancer using The Cancer Genome Atlas data and single-cell RNA sequencing of human breast cancer, as well as the association of gland reorganization to different breast cancer subtypes. This comprehensive murine mammary cell gene expression atlas (https://mouse-mammary-epithelium-integrated.cells.ucsc.edu) presented novel insights into the impact of the internal and external stimuli on the mammary epithelium at an advanced resolution. This dataset includes all the relevant scripts and data associated with the publication in the Communications Biology journal.