The aim of this paper is to estimate the ego-motion of an RGB-D camera in dynamic environments. A semi-direct motion estimation pipeline is modified for the RGB-D camera. In order to avoid the impact of dynamic objects, a new mapping method based on scoring mechanism is proposed, which can effectively remove feature points on dynamic objects and results a map contains only static points. The method is evaluated not only with the TUM RGB-D benchmark but also using an Asus Xtion Pro Live camera in a dynamic office environment. The experimental results show that our method has higher accuracy in dynamic environments and has considerable accuracy in static environments. In some high dynamic scenes, the accuracy of our method is more than 7 times higher than other RGB-D visual odometry algorithms.