The main purpose of this paper is to investigate the parallelism of vectors in homogeneous spaces. The definition of a vector and the condition for spaces under which a covariant differential of a vector is also a vector were given by E. Cartan [4] in a very intuitive way. Here I formulate this in a stricter way by his method of moving frame. Even if a homogeneous space has the property that the covariant differential of a vector is of the same kind, another definition of covariant differential may also have the required property. I will give a necessary and sufficient condition under which the definition of covariant differential is unique. Once the covariant differential has been defined it is easy to introduce a parallelism of vectors in the space. But the parallelism depends in general on the path along which we translate a vector. The condition for the spaces with an absolute parallelism can be obtained.