Estrogen receptor alpha (ERα, NR3A1) contributes through its expression in different tissues to a spectrum of physiological processes, including reproductive system development and physiology, bone mass maintenance, as well as cardiovascular and central nervous system functions. It is also one of the main drivers of tumorigenesis in breast and uterine cancer and can be targeted by several types of hormonal therapies. ERα is expressed in a subset of luminal cells corresponding to less than 10% of normal mammary epithelial cells and in over 70% of breast tumors (ER+ tumors), but the basis for its selective expression in normal or cancer tissues remains incompletely understood. The mapping of alternative promoters and regulatory elements has delineated the complex genomic structure of the ESR1 gene and shed light on the mechanistic basis for the tissue-specific regulation of ESR1 expression. However, much remains to be uncovered to better understand how ESR1 expression is regulated in breast cancer. This review recapitulates the current body of knowledge on the structure of the ESR1 gene and the complex mechanisms controlling its expression in breast tumors. In particular, we discuss the impact of genetic alterations, chromatin modifications, and enhanced expression of other luminal transcription regulators on ESR1 expression in tumor cells.