With digital photo and video cameras currently taking an indispensable part of many domestic households, a vast amount of research and software development has been aimed towards tools for facilitating the manipulation of images and videos. Ranging from foreground segmentation to deblurring, many powerful tools have been made available. In this dissertation we mainly focus on the use of video sprites in so called image- and video-based animation, synthesis and augmentation techniques. These techniques use one or more images as input, analyse this data and animate, synthesize or augment it to obtain new images inspired by the original. The best known video-based synthesis example is the Video Textures algorithm from Schödl et al. [Schödl 00b]. Starting from a relatively short video sequence, a new video is created by rearranging the input frames in such a way that the resulting video can be infinitely looped while adhering to the appearance of the original content and avoiding visually harsh transitions. An important extension to this work was provided in 2002 with the Video Sprites technique of Schödl et al. [Schödl 02]. Instead of playing back whole frames, images of a character are extracted from video and concatenated to form new animations. Where video textures are mostly limited to work with videos that inherently feature repetitive visual content like natural phenomena, the video sprites algorithm allows the animation of small characters like hamsters or flies. With both techniques allowing only a limited amount of possible input videos, we introduce two video based animation and synthesis techniques that are aimed at videos containing different kinds of subjects.