This paper contributes to the literature considering the wage effects of educational mismatch in Germany. It uses a large German panel data set for the period from 1984 to 1997 and stresses the importance of controlling for unobserved heterogeneity when analyzing the labor market effects of over- and undereducation. Using pooled OLS, the estimation results confirm those found in the existing literature. The estimated differences between adequately and inadequately educated workers become smaller or disappear totally, when controlling for unobserved heterogeneity. In contrast to the common results using only cross-sectional data, the human capital model could not be rejected once controlling for unobserved heterogeneity.