SUMMARY Significant changes have occurred in plant cell wall composition during evolution and diversification of tracheophytes. As the sister lineage to seed plants, knowledge on the cell wall of ferns is key to track evolutionary changes across tracheophytes and to understand seed plant‐specific evolutionary innovations. Fern cell wall composition is not fully understood, including limited knowledge of glycoproteins such as the fern arabinogalactan proteins (AGPs). Here, we characterize the AGPs from the leptosporangiate fern genera Azolla , Salvinia , and Ceratopteris . The carbohydrate moiety of seed plant AGPs consists of a galactan backbone including mainly 1,3‐ and 1,3,6‐linked pyranosidic galactose, which is conserved across the investigated fern AGPs. Yet, unlike AGPs of angiosperms, those of ferns contained the unusual sugar 3‐ O ‐methylrhamnose. Besides terminal furanosidic arabinose, Ara (Ara f ), the main linkage type of Ara f in the ferns was 1,2‐linked Ara f , whereas in seed plants 1,5‐linked Ara f is often dominating. Antibodies directed against carbohydrate epitopes of AGPs supported the structural differences between AGPs of ferns and seed plants. Comparison of AGP linkage types across the streptophyte lineage showed that angiosperms have rather conserved monosaccharide linkage types; by contrast bryophytes, ferns, and gymnosperms showed more variability. Phylogenetic analyses of glycosyltransferases involved in AGP biosynthesis and bioinformatic search for AGP protein backbones revealed a versatile genetic toolkit for AGP complexity in ferns. Our data reveal important differences across AGP diversity of which the functional significance is unknown. This diversity sheds light on the evolution of the hallmark feature of tracheophytes: their elaborate cell walls.