In plants, two atypical DNA-dependent RNA polymerases, RNA polymerase IV (Pol IV) and Pol V, and an RNA-DEPENDENT RNA POLYMERASE 2 (RDR2) together produce noncoding RNAs (ncRNAs) to guide the plant-specific RNA-directed DNA methylation (RdDM). Although both Pol IV and Pol V have evolved from the canonical Pol II, they have adapted to different roles in RdDM. The mechanisms of their adaptation are key to understanding plant DNA methylation and the divergent evolution of polymerases. In this review, we summarize insights that have emerged from recent structural studies of Pol IV, Pol V, and RDR2 and discuss their structural features critical for efficient ncRNA production in RdDM.