Human milk oligosaccharides (HMOs) comprise a group of structurally complex, unconjugated glycans that are highly abundant in human milk. HMOs are minimally digested in the gastrointestinal tract and reach the colon intact, where they shape the microbiota. A small fraction of HMOs is absorbed, reaches the systemic circulation, and is excreted in urine. HMOs can bind to cell surface receptors expressed on epithelial cells and cells of the immune system and thus modulate neonatal immunity in the infant gut, and possibly also sites throughout the body. In addition, they have been shown to act as soluble decoy receptors to block the attachment of various microbial pathogens to cells. This review summarizes the current knowledge of the effects HMOs can have on infections, allergies, auto-immune diseases and inflammation, and will focus on the role of HMOs in altering immune responses through binding to immune-related receptors.