Abstract In this paper we present an approach for action recognition that uses various applications of Fourier transform. The main idea is to classify video sequences based on action representations obtained using shape descriptors. For shape representation we use the Two-Dimensional Fourier Descriptor, Generic Fourier Descriptor and UNL-Fourier Descriptor. For each sequence of binary silhouettes we derive a set of shape descriptors and match the descriptor of the first frame with the rest of descriptors to obtain a vector of similarities (correlation coefficients or C1 correlations) or dissimilarities (Euclidean distances). Then normalized vectors are transformed into action representations using discrete Fourier transform, power spectral density estimate or a combination of both. Classification is performed using leave-one-out cross-validation and various matching measures. Additionally, we incorporate a coarse classification step that distinguish between actions performed in place and actions with changing location of a silhouette. Extensive experiments are carried out to investigate all possible combinations of selected processing steps, and experimental results are promising.