Regulation of plant cell wall deposition and patterning is essential for the normal growth and development of plants. Small GTPases play pivotal roles in the modulation of primary cell wall formation by controlling cytoskeletal organization and membrane trafficking. However, the functions of small GTPases in secondary cell wall development are poorly understood. Recent studies on xylem cells revealed that the Rho of plants (ROP) group of small GTPases critically participates in the spatial patterning of secondary cell walls. In differentiating xylem cells, a specific GTPase-activating protein (GAP)/guanine nucleotide exchange factor (GEF) pair facilitates local activation of ROP11 to establish de novo plasma membrane domains. The activated ROP11 then recruits a microtubule-associated protein, MIDD1, to mediate the mutual inhibition between cortical microtubules and active ROP. Furthermore, recent works suggest that certain small GTPases, including ROP and Rab GTPases, regulate membrane trafficking to establish secondary cell wall deposition and patterning. Accordingly, this mini-review assesses and summarizes the current literature regarding the emerging functions of small GTPases in the development of secondary cell walls.