We study the existence of Riemannian metrics with zero topological entropy on a closed manifold M with infinite fundamental group. We show that such a metric does not exist if there is a finite simply connected CW complex which maps to M in such a way that the rank of the map induced in the pointed loop space homology grows exponentially. This result allows us to prove in dimensions four and five, that if M admits a metric with zero entropy then its universal covering has the rational homotopy type of a finite elliptic CW complex. We conjecture that this is the case in every dimension.