The interaction between cellulose and xylan is important for the load-bearing secondary cell wall of flowering plants. Based on the precise, evenly spaced pattern of acetyl and glucuronosyl (MeGlcA) xylan substitutions in eudicots, we recently proposed that an unsubstituted face of xylan in a 2-fold helical screw can hydrogen bond to the hydrophilic surfaces of cellulose microfibrils. In gymnosperm cell walls, any role for xylan is unclear, and glucomannan is thought to be the important cellulose-binding polysaccharide. Here, we analyzed xylan from the secondary cell walls of the four gymnosperm lineages (Conifer, Gingko, Cycad, and Gnetophyta). Conifer, Gingko, and Cycad xylan lacks acetylation but is modified by arabinose and MeGlcA. Interestingly, the arabinosyl substitutions are located two xylosyl residues from MeGlcA, which is itself placed precisely on every sixth xylosyl residue. Notably, the Gnetophyta xylan is more akin to early-branching angiosperms and eudicot xylan, lacking arabinose but possessing acetylation on alternate xylosyl residues. All these precise substitution patterns are compatible with gymnosperm xylan binding to hydrophilic surfaces of cellulose. Molecular dynamics simulations support the stable binding of 2-fold screw conifer xylan to the hydrophilic face of cellulose microfibrils. Moreover, the binding of multiple xylan chains to adjacent planes of the cellulose fibril stabilizes the interaction further. Our results show that the type of xylan substitution varies, but an even pattern of xylan substitution is maintained among vascular plants. This suggests that 2-fold screw xylan binds hydrophilic faces of cellulose in eudicots, early-branching angiosperm, and gymnosperm cell walls.