Environment mapping, or reflection mapping, has been widely used in the game and movie industries to give objects a realistic illumination atmosphere. For moving objects, direct frame-by-frame calculation of environment maps and correspondence-based interpolation are both impractical for real-time applications due to the large computational costs. To deal with this problem, "fake" environment mapping with a fixed, pre-generated environment image has been commonly used, but clearly such an approximation is inadequate for a highly reflective object whose environment is constantly changing as it moves. In this paper, we present an approach that sparsely samples environment maps of a moving object and rapidly interpolates them for high performance. Two techniques are introduced for fast environment map interpolation without computation of scene shading. The first method utilizes scene geometry to facilitate interpolation, and the second involves geometry reconstruction from depth buffer values to reduce inefficiencies caused by complex scene geometry. These two techniques can easily be implemented in graphics hardware, and test results show that they achieve significant boosts in performance over frame-by-frame environment map computation with little loss in visual quality.