We present optical spectroscopy of 62 objects selected from several samples of ultra steep spectrum (USS) radio sources. 46 of these are from our primary catalog, consisting of 669 sources with radio spectral indices alpha < -1.30 (S_nu ~ nu^alpha); this first spectroscopic sub-sample was selected on the basis of their faint optical and near-IR identifications. Most are identified as narrow-lined radio galaxies with redshifts ranging from z=0.25 to z=5.19. Ten objects are at z>3, nearly doubling the number of such sources known to date. Four of the USS radio sources are identified with quasars, of which at least three have very red spectral energy distributions. The source TN J0936-2242 is identified with an extremely red object (ERO, R-K>5); both it and a close companion are at z=1.479. The spectrum of the ERO closely resembles that of previously discovered radio galaxies at z~1.5. Five sources show continuum emission, but fail to show any clear emission or absorption features, despite integrations of ~1 h with the Keck telescope. We suggest that these objects could be (i) radio galaxies with faint emission lines in the ``redshift desert'' at 1.5 7 candidates.

59 Pages, including 59 PostScript figures. Accepted for publication in the Astronomical Journal (March 2001 issue). Tables 5 and 6 will be available in electronic format on the AJ website or upon request