Non-cellulosic polysaccharides constitute approximately one third of usable woody biomass for human exploitation. In contrast to cellulose, these substances are composed of several different types of unit monosaccharides and their backbones are substituted by various groups. Their structural diversity and recent examples of their modification in transgenic plants and mutants suggest they can be targeted for improving wood-processing properties, thereby facilitating conversion of wood in a biorefinery setting. Critical knowledge on their structure-function relationship is slowly emerging, although our understanding of molecular interactions responsible for observed phenomena is still incomplete. This review: (1) provides an overview of structural features of major non-cellulosic polysaccharides of wood, (2) describes the fate of non-cellulosic polysaccharides during biorefinery processing, (3) shows how the non-cellulosic polysaccharides impact lignocellulose processing focused on yields of either sugars or polymers, and (4) discusses outlooks for the improvement of tree species for biorefinery by modifying the structure of non-cellulosic polysaccharides.