In action recognition, a body pose, or simply pose, is defined as a state of the 3D position of skeleton joints which varies while an action is performed. Then an action can be defined as a sequence of specific poses, called key-poses in this paper. Key-poses are contained in the so called key-frames, which are surrounded by several other frames. Key-frames can be selected using minimum energy criterion. In this paper, a method is proposed to automatically extract key-poses out of key-frames. Furthermore, a novel learning method, named Fisher forest, is proposed which enables action recognition methods to use different types and even various number of joints for different actions, poses, or situations. The proposed Fisher forest method can also be used as an effective classifier for classifying vectors of classes with different essences and even different dimensions, and thus can be used in other applications which use data with different types or dimensions in several conditions. In this work, Fisher forest is utilized for using key-joints in action recognition. The proposed key-poses and key-joints (using Fisher forest) methods are in the category of spatio-temporal techniques, which are proposed here for the data of skeleton in action recognition. Different experiments verify the effectiveness of proposed techniques.