Glutamate carboxypeptidase II (GCPII) is a membrane‐bound cell surface peptidase. There is significant interest in the inhibition of GCPII as a means of neuroprotection, while GCPII inhibition as a method to treat prostate cancer remains a topic of further investigation. The key zinc‐binding functional group of the well‐characterized classes of GCPII inhibitors (phosphonates and phosphoramidates) is tetrahedral and negatively charged at neutral pH, while glutamyl urea class of inhibitors possesses a planar and neutral zinc‐binding group. This study explores a new class of GCPII inhibitors, glutamyl sulfamides, which possess a putative net neutral tetrahedral zinc‐binding motif. A small library containing six sulfamides was prepared and evaluated for inhibitory potency against purified GCPII in an enzymatic assay. While most inhibitors have potencies in the micromolar range, one showed promising sub‐micromolar potency, with the optimal inhibitor in this series being aspartyl–glutamyl sulfamide (2d). Lastly, computational docking was used to develop a tentative binding model on how the most potent inhibitors interact with the ligand‐binding site of GCPII.