We describe, in an intrinsic way and using the global chart provided by Ito's parallel transport, a generalisation of the notion of geodesic (as critical path of an energy functional) to diffusion processes on Riemannian manifolds. These stochastic processes are no longer smooth paths but they are still critical points of a regularised stochastic energy functional. We consider stochastic geodesics on compact Riemannian manifolds and also on (possibly infinite dimensional) Lie groups. Finally the question of existence of such stochastic geodesics is discussed: we show how it can be approached via forward-backward stochastic differential equations.