Learning to reason in large-scale knowledge graphs has attracted much attention from research communities recently. This paper targets a practical task of multi-hop reasoning in knowledge graphs, which can be applied in various downstream tasks such as question answering, and recommender systems. A key challenge in multi-hop reasoning is to synthesize structural information (e.g., paths) in knowledge graphs to perform deeper reasoning. Existing methods usually focus on connection paths between each entity pair. However, these methods ignore predecessor paths before connection paths and regard entities and relations within every single path as equally important. With our observations, predecessor paths before connection paths can provide more accurate semantic representations. Furthermore, entities and relations in a single path contribute variously to the right answers. To this end, we propose a novel model HiAM (Hierarchical Attention based Model) for knowledge graph multi-hop reasoning. HiAM makes use of predecessor paths to provide more accurate semantics for entities and explores the effects of different granularities. Firstly, we extract predecessor paths of head entities and connection paths between each entity pair. Then, a hierarchical attention mechanism is designed to capture the information of different granularities, including entity/relation-level and path-level features. Finally, multi-granularity features are fused together to predict the right answers. We go one step further to select the most significant path as the explanation for predicted answers. Comprehensive experimental results demonstrate that our method achieves competitive performance compared with the baselines on three benchmark datasets.