In this paper, we present a novel method for human action recognition using covariance features. Computationally efficient action features are extracted from the skeleton of the subject performing the action. They aim to capture relative positions of the joints and motion over time. These features are encoded into a compact representation using a covariance matrix. We evaluate the performance of the proposed method and demonstrate its superiority compared to related state-of-the-art methods on various datasets, including the MSR Action 3D, the MSR Daily Activity 3D and the UTKinect-Action dataset.