Semi-fragile video watermarking is a technology for detecting manipulations. It provides robustness against content-preserving manipulations as well as sensitivity to content-changing manipulations. To achieve this, robust content-describing features are applied. We use the SIFT keypoint detection as feature for our semifragile video watermarking scheme introduced in this work. SIFT (Scale Invariant Feature Transformation) detects points invariant to image scale and rotation and can be used for object matching after changing the 3D viewpoint, addition of noise and modifications in illumination. With the detected feature points we generate an authentication message, which is embedded with a robust video watermark. In the verification process we introduce a temporal filtering approach to reduce the distortions caused by content-preserving manipulations. We present experimental results demonstrating the robustness and sensitivity of our scheme.