Abstract Global motion estimation, being one of the most important tools in video processing field with many applications, is mainly carried out in pixel or compressed domain. Since those based on the pixels have drawbacks such as high computational complexity, most researches are oriented to the compressed domain in which motion vectors are utilized. On the other hand, there are many unwanted existing outliers in motion vector based global motion estimation because of noise or foreground effects. In this paper, proposed motion vector dissimilarity measure is used to remove the outliers to provide fast and accurate motion vector based global motion estimation. Performance of the dissimilarity measure is further improved by using different neighborhood orientations. Also phase correlation of motion vectors are effectively utilized. Therefore small noisy motion vectors are easily detected and different orientations contribute to both performance and low latency. Experiments using the proposed method achieve more accurate results with less computational complexity compared to the state of the art methods.