Pedestrian detection is a challenging problem studied over decades. Most algorithms are based on human appearance. Only few works consider motion as a feature component. In this paper, however, we tackle this problem only considering short periods of pedestrian walking. This motion does not depend on the variations of pedestrian pose, body shape, illumination, and background. We model pedestrian motion that has unique properties compare to background and rigid objects motion in spatial-temporal motion profiles. This observation helps us to identify pedestrian leg motion along with body motion over a short time period. Our method also works for a vehicle borne camera where background also moves. We achieved more robust results by dealing with crowds, and other degenerating cases of human motion against background and dynamic scenes. The method has a low computational cost on a motion profile and it can be combined with a shape-based method as pre-screening for reducing the false positives. It also provides a feasible way to find human behaviors.