Human milk oligosaccharides have been recognized as an important, functional biomolecule in mothers’ milk. Moreover, these oligosaccharides have been recognized as the third most abundant component of human milk, ranging from 10–15 g/L in mature milk and up to and over 20 g/L reported in colostrum. Initially, health benefits of human milk oligosaccharides were assigned via observational studies on the differences between breastfed and bottle fed infants. Later, pools of milk oligosaccharides were isolated and used in functional studies and in recent years more specific studies into structure–function relationships have identified some advanced roles for milk oligosaccharides in the healthy development of infants. In other research, the levels, diversity, and complexity of human milk oligosaccharides have been studied, showing a wide variation in results. This review gives a critical overview of challenges in the analysis of human milk oligosaccharides. In view of the myriad functions that can be assigned, often to specific structures or classes of structures, it is very relevant to assess the levels of these structures in the human milk correctly, as well as in other biological sample materials. Ultimately, the review makes a case for a comparative, inter-laboratory study on quantitative human milk oligosaccharide analysis in all relevant biological samples.