Only a few breast cancer histologic subtypes harbor distinct genetic alterations that are associated with a specific morphology (genotype-phenotype correlation). Secretory carcinomas and adenoid cystic carcinomas are each characterized by recurrent translocations, and invasive lobular carcinomas frequently have CDH1 mutations. Solid papillary carcinoma with reverse polarity is a rare breast cancer subtype with a distinctive morphology and recently identified IDH2 mutations. We review the clinical and pathologic features and underlying genetic alterations of those breast cancer subtypes with established genotype-phenotype correlations and discuss the phenotypes associated with germline mutations in genes associated with hereditary breast cancer.