Let \(M\) be a compact Riemannian manifold with negative Ricci curvature. The author shows that if the universal cover of \(M\) has a pole and if any geodesic sphere centered at the pole is convex or concave, then the growth function of the fundamental group is at least exponential. The author also shows that if \(M\) is a complete Riemannian manifold with asymptotically non-negative Ricci curvature, then any finitely generated subgroup of the fundamental group has at least polynomial growth. These results generalize previous results of Milnor.