We study the dynamics of a probe particle driven by a constant force through a colloidal glass of hard spheres. This nonequilibrium and anisotropic problem is investigated using a new implementation of the mode-coupling approximation with multiple relaxation channels and Langevin dynamics simulations. A force threshold is found, below which the probe remains localized, while above it the probe acquires a finite velocity. We focus on the localized regime, comparing theory and simulations concerning the dynamics in the length scale of the cage and the properties of the transition to the delocalized regime, such as the critical power-law decay of the probe correlation function. Probe van Hove functions predicted by the theory show exponential tails reminiscent of an intermittent dynamics of the probe. This scenario is microscopically supported by simulations.