(Abridged) We present R~600, 10-37um spectra of 53 ULIRGs at z<0.32, taken using the IRS on board Spitzer. All of the spectra show fine structure emission lines of Ne, O, S, Si and Ar, as well as molecular Hydrogen lines. Some ULIRGs also show emission lines of Cl, Fe, P, and atomic Hydrogen, and/or absorption features from C_2H_2, HCN, and OH. We employ diagnostics based on the fine-structure lines, as well as the EWs and luminosities of PAH features and the strength of the 9.7um silicate absorption feature (S_sil), to explore the power source behind the infrared emission in ULIRGs. We show that the IR emission from the majority of ULIRGs is powered mostly by star formation, with only ~20% of ULIRGs hosting an AGN with a comparable or greater IR luminosity than the starburst. The detection of the 14.32um [NeV] line in just under half the sample however implies that an AGN contributes significantly to the mid-IR flux in ~42% of ULIRGs. The emission line ratios, luminosities and PAH EWs are consistent with the starbursts and AGN in ULIRGs being more extincted, and for the starbursts more compact, versions of those in lower luminosity systems. The excitations and electron densities in the NLRs of ULIRGs appear comparable to those of lower luminosity starbursts, though there is evidence that the NLR gas in ULIRGs is more dense. We show that the combined luminosity of the 12.81um [NeII] and 15.56um [NeIII] lines correlates with both IR luminosity and the luminosity of the 6.2 micron and 11.2 micron PAH features in ULIRGs, and use this to derive a calibration between PAH luminosity and star formation rate. Finally, we show that ULIRGs with 0.8 < S_sil < 2.4 are likely to be powered mainly by star formation, but that ULIRGs with S_sil < 0.8, and possibly those with S_sil > 2.4, contain an IR-luminous AGN.

62 pages in preprint format, 4 tables, 23 figures. ApJ accepted