SummaryThe transition of C3, via C2 towards C4 photosynthesis is an important example of stepwise evolution of a complex genetic trait. A common feature that was gradually emphasized during this trajectory is the evolution of a CO2 concentration mechanism around Rubisco. In C2 plants, this mechanism is based on tissue‐specific accumulation of glycine decarboxylase (GDC) in bundle sheath (BS) cells, relative to global expression in the cells of C3 leaves. This limits photorespiratory CO2 release to BS cells. Because BS cells are surrounded by photosynthetically active mesophyll cells, this arrangement enhances the probability of re‐fixation of CO2. The restriction of GDC to BS cells was mainly achieved by confinement of its P‐subunit (GLDP). Here, we provide a mechanism for the establishment of C2‐type gene expression by studying the upstream sequences of C3 Gldp genes in Arabidopsis thaliana. Deletion of 59 bp in the upstream region of AtGldp1 restricted expression of a reporter gene to BS cells and the vasculature without affecting diurnal variation. This region was named the ‘M box’. Similar results were obtained for the AtGldp2 gene. Fusion of the M box to endogenous or exogenous promoters supported mesophyll expression. Nucleosome densities at the M box were low, suggesting an open chromatin structure facilitating transcription factor binding. In silico analysis defined a possible consensus for the element that was conserved across the Brassicaceae, but not in Moricandia nitens, a C2 plant. Collective results provide evidence that a simple mutation is sufficient for establishment of C2‐specific gene expression in a C3 plant.