Perturbative expansions of relativistic quantum field theories typically contain ultraviolet divergences requiring regularization and renormalization. Many different regularization techniques have been developed over the years, but most regularizations require severe mutilation of the logical foundations of the theory. In contrast, breaking Lorentz invariance, while it is certainly a radical step, at least does not damage the logical foundations of the theory. We shall explore the features of a Lorentz symmetry breaking regulator in a simple polynomial scalar field theory, and discuss its implications. We shall quantify just "how much" Lorentz symmetry breaking is required to fully regulate the theory and render it finite. This scalar field theory provides a simple way of understanding many of the key features of Horava's recent article [arXiv:0901.3775 [hep-th]] on 3+1 dimensional quantum gravity.

15 pages; 3 figures; V2: five references added, two minor typos fixed. no change in physics; V3: seven pages, zero figures, various references added and updated, discussion somewhat streamlined. no change in physics. This version accepted for publication in Physical Review D