We study first order quantum phase transitions in mean-field spin glasses. We solve the quantum Random Energy Model using elementary methods and show that at the transition the eigenstate suddenly projects onto the unperturbed ground state and that the gap between the lowest states is exponentially small in the system size. We argue that this is a generic feature of all `Random First Order' models, which includes benchmarks such as random satisfiability. We introduce a two-time instanton to calculate this gap in general, and discuss the consequences for quantum annealing.

4 pages, 3 figures, minor typos corrected