Recognizing human actions is of vital interest in video surveillance or ambient assisted living. We consider an action as a sequence of body poses which are themselves a linear combination of body parts. In an offline procedure, nonnegative tensor factorization is used to extract basis images that represent body parts. The weighting coefficients are obtained by filtering a frame with the set of basis images. Since the basis images are obtained from nonnegative tensor factorization, they are separable and filtering can be implemented efficiently. The weighting coefficients encode dynamics and are used for action recognition. In the proposed action recognition framework, neither explicit detection and tracking of humans nor background subtraction are needed. Furthermore, for recognizing location specific actions, we implicitely take scene objects into account.