Type 2 diabetes is a complex metabolic disease with hyperglycemia as its recognizable hallmark. Hepatic glucose output is elevated in Type 2 diabetic patients, and evidence suggests drugs which lower hepatic glucose production are effective antihyperglycemic agents. Glycogenolysis, which is the release of monomeric glucose from its polymeric storage form called glycogen, is a key contributor to hepatic glucose output. Glycogen phosphorylase is the enzyme that catalyzes this process. This review covers advances in the design of small molecule inhibitors of this enzyme, their biological activity, and their potential as effective antihyperglycemic agents for the treatment of Type 2 diabetes.