Action is any meaningful movement of the human and it is used to convey information or to interact naturally without any mechanical devices. Human action recognition is motivated by some of the applications such as video retrieval, Human robot interaction, to interact with deaf and dumb people etc. In any Action Recognition System, some pre-processing steps are done for removing the noise caused because of illumination effects, blurring, false contour etc. Background subtraction is done to remove the static or slowly varying background. In this paper, multiple background subtraction algorithms are tested and then one of them is selected for the further process of action recognition. Background subtraction is also known as foreground/background segmentation or foreground extraction. The next step is the feature extraction which deals with the extraction of the important feature (like corner points, optical flow, shape, motion vectors etc.) from the image frame. The proposed novel action recognition algorithm uses discrete Fourier transform (DFT) of the small image block.