Phosphodiesterases (PDEs) comprise a family of enzymes that modulate the immune response, inflammation, and memory, among many other functions. There are three types of PDEs: cAMP-specific, cGMP-specific, and dual-specific. Here we describe the mechanism of nucleotide selectivity on the basis of high-resolution co-crystal structures of the cAMP-specific PDE4B and PDE4D with AMP, the cGMP-specific PDE5A with GMP, and the apo-structure of the dual-specific PDE1B. These structures show that an invariant glutamine functions as the key specificity determinant by a "glutamine switch" mechanism for recognizing the purine moiety in cAMP or cGMP. The surrounding residues anchor the glutamine residue in different orientations for cAMP and for cGMP. The PDE1B structure shows that in dual-specific PDEs a key histidine residue may enable the invariant glutamine to toggle between cAMP and cGMP. The structural understanding of nucleotide binding enables the design of new PDE inhibitors that may treat diseases in which cyclic nucleotides play a critical role.