Methylation of histones has been regarded as a stable modification defining the epigenetic program of the cell, which regulates chromatin structure and transcription. However, the recent discovery of histone demethylases has challenged the stable nature of histone methylation. Here we demonstrate that the JARID1 proteins RBP2, PLU1, and SMCX are histone demethylases specific for di- and trimethylated histone 3 lysine 4 (H3K4). Consistent with a role for the JARID1 Drosophila homolog Lid in regulating expression of homeotic genes during development, we show that RBP2 is displaced from Hox genes during embryonic stem (ES) cell differentiation correlating with an increase of their H3K4me3 levels and expression. Furthermore, we show that mutation or RNAi depletion of the C. elegans JARID1 homolog rbr-2 leads to increased levels of H3K4me3 during larval development and defects in vulva formation. Taken together, these results suggest that H3K4me3/me2 demethylation regulated by the JARID1 family plays an important role during development.