Precession of magnetization via the inverse Faraday effect is investigated with a view of determining the fundamental limit on the precession speed. Such a limit could have important consequences for ultrafast magnetic switching. The angular momentum required for precession is shown to be supplied by the light. This indicates that there is no fundamental obstruction to magnetization reversal on the time scale of a laser pulse provided that a material with a sufficiently strong magneto-optical response can be found.