With the development and popularity of powerful video editing tools, it tends to be increasingly easier to create digital synthesized videos. A novel passive video inter-frame forgery detection method based on nonnegative tensor factorization (NTF) is presented in this work. It is based on the finding that inter-frame forgery will disturb the consistency of time-dimension factor. In this method, the video is factorized by using NTF first and then the time-dimension factor is extracted. By comparing the correlation between the elements of the factor, the video forgery can be detected. According to the experimental results, it shows that the proposed scheme is able to expose frame deletion and insertion forgery effectively.