Animals use their whole body to navigate their environment and solve tasks. Identification of common dynamics of body parts using quantitative methods is key to study underlying motor system organization. Zebrafish (Danio rerio) need to coordinate tail and pectoral fin movements and can serve as a model organism for the study of locomotion generation. Behavioral tracking of larval zebrafish has usually restricted itself either to track only the position and tail movements of freely swimming zebrafish larvae, or to track fine movements of body parts in immobilized fish. This precludes studies that investigate coordination or postural control. To study full body coordination, we built a setup that allows us to track freely swimming larval zebrafish, while acquiring high resolution images of their whole body. This enables tracking of both tail and fins using DeepLabCut with additional segmentation and classification of the tail using the Megabouts algorithm. First, we describe stereotypical fin dynamics at bout onset which show differences between modes of movements as forward swims and turns. Thereafter we show that fin dynamics can be embedded on a low dimensional manifold with axes corresponding to amplitude, vigor and frequency. Based on this embedding, fin dynamics show a degree of stereotypy with associated tail classification. Lastly, we show that a model that captures the latent dynamics underlying tail movement generation, developed using ILQR-VAE, is able to predict fin dynamics, hinting towards a coupled control mechanism. Overall, this represents an important step towards building a comprehensive dictionary of larval zebrafish locomotion that will enable better study of motion generation and coordination.