This paper performs an asymptotic analysis of penalized spline estimators. We compare P-splines and splines with a penalty of the type used with smoothing splines. The asymptotic rates of the supremum norm of the difference between these two estimators over compact subsets of the interior and over the entire interval are established. It is shown that a P-spline and a smoothing spline are asymptotically equivalent provided that the number of knots of the P-spline is large enough, and the two estimators have the same equivalent kernels for both interior points and boundary points.