In this paper, we study homogeneous geodesics in homogeneous Finsler spaces. We first give a simple criterion that characterizes geodesic vectors. We show that the geodesics on a Lie group, relative to a bi-invariant Finsler metric, are the cosets of the one-parameter subgroups. The existence of infinitely many homogeneous geodesics on compact semi-simple Lie group is established. We introduce the notion of naturally reductive homogeneous Finsler space. As a special case, we study homogeneous geodesics in homogeneous Randers spaces. Finally, we study some curvature properties of homogeneous geodesics. In particular, we prove that the S-curvature vanishes along the homogeneous geodesics.