The identification of histone methyltransferases and demethylases has uncovered a dynamic methylation system needed to modulate appropriate levels of gene expression. Gene expression levels of various histone demethylases, such as the JARID1 family, show distinct patterns of embryonic and adult expression and respond to different environmental cues, suggesting that histone demethylase protein levels must be tightly regulated for proper development. In our study, we show that the protein level of the yeast histone H3 Lys 4 (H3 K4) demethylase Jhd2/Kdm5 is modulated through polyubiquitination by the E3 ubiquitin ligase Not4 and turnover by the proteasome. We determine that polyubiquitin-mediated degradation of Jhd2 controls in vivo H3 K4 trimethylation and gene expression levels. Finally, we show that human NOT4 can polyubiquitinate human JARID1C/SMCX, a homolog of Jhd2, suggesting that this is likely a conserved mechanism. We propose that Not4 is an E3 ubiquitin ligase that monitors and controls a precise amount of Jhd2 protein so that the proper balance between histone demethylase and histone methyltransferase activities occur in the cell, ensuring appropriate levels of H3 K4 trimethylation and gene expression.