Fat is the main energy providing component in human milk and comprising a complex mixture of different lipid species, with quantitative dominance of triglycerides. After elucidating the fatty acid composition, more recent research has looked at influencing factors and the importance of specific lipids. Here we review quantitative aspects of maternal metabolism which contribute to the milk fatty acid composition, especially considering essential fatty acids and their long chain polyunsaturated derivatives. In this context studies with stable isotopes have indicated the importance of maternal body pools for mediating the effects of diet on milk composition. Furthermore, the importance of positioning of palmitic acid at the glycerol backbone of triglycerides is discussed, and the phospholipids of the milk fat globule membrane are described and examples for their potential importance for infant development are presented.