We study the existence of extremal Kähler metrics on Kähler manifolds. After introducing a notion of relative K-stability for Kahler manifolds, we prove that Kähler manifolds admitting extremal Kähler metrics are relatively K-stable. Along the way, we prove a general Lp lower bound on the Calabi functional involving test configurations and their associated numerical invariants, answering a question of Donaldson. When the Kähler manifold is projective, our definition of relative K-stability is stronger than the usual definition given by Székelyhidi. In particular our result strengthens the known results in the projective case (even for constant scalar curvature Kähler metrics), and rules out a well known counterexample to the "naïve" version of the Yau-Tian-Donaldson conjecture in this setting.