Ionizing radiation-associated breast cancer risk appears to be modified by timing of reproductive events such as age at radiation exposure, parity, age at first live birth, and age at menopause. However, potential breast cancer risk modification of low to moderate radiation dose by polymorphic estrogen metabolism-related gene variants has not been routinely investigated. We assessed breast cancer risk of 12 candidate variants in 12 genes involved in steroid metabolism, catabolism, binding, or receptor functions in a study of 859 cases and 1,083 controls within the US radiologic technologists (USRT) cohort. Using cumulative breast dose estimates from a detailed assessment of occupational and personal diagnostic ionizing radiation exposure, we investigated the joint effects of genotype on the risk of breast cancer. In multivariate analyses, we observed a significantly decreased risk of breast cancer associated with the CYP3A4 M445T minor allele (rs4986910, OR = 0.3; 95% CI 0.1-0.9). We found a borderline increased breast cancer risk with having both minor alleles of CYP1B1 V432L (rs1056836, CC vs. GG, OR = 1.2; 95% CI 0.9-1.6). Assuming a recessive model, the minor allele of CYP1B1 V432L significantly increased the dose-response relationship between personal diagnostic X-ray exposure and breast cancer risk, adjusted for cumulative occupational radiation dose (p (interaction) = 0.03) and had a similar joint effect for cumulative occupational radiation dose adjusted for personal diagnostic X-ray exposure (p (interaction) = 0.06). We found suggestive evidence that common variants in selected estrogen metabolizing genes may modify the association between ionizing radiation exposure and breast cancer risk.