ABSTRACTWe present an automatic, efficient, and simple technique to create pencil drawing animation, starting from a video. We generate pencil drawing from a source frame based on stroke modeling, specifying the properties of strokes, in combination with layered lines, flow‐guided difference‐of‐Gaussian (DoG) filter to several layers. Generated pencil strokes are translated and rotated because of forces exerted from the sequential frames using rigid body dynamics. Linear and angular forces acting on strokes are calculated according to the temporally filtered per‐pixel optical flow vectors. Our framework effectively generates the coherent animation of pencil strokes preserving the structured appearance of charcoal or pastel, which is difficult to achieve with previous‐abstraction based non‐photorealistic animation. Moreover, our stroke simulation step is suitable for animating the different styles of strokes, such as oil painting and watercolor, and can be efficiently implemented to produce animation in relatively inexpensive manner. Copyright © 2013 John Wiley & Sons, Ltd.