Lagged variables are often used as instruments when the generalized method of moments (GMM) is applied to time series data. We show that if these variables follow noncausal autoregressive processes, their lags are not valid instruments and the GMM estimator is inconsistent. Moreover, in this case, endogeneity of the instruments may not be revealed by the J-test of overidentifying restrictions that may be inconsistent and, as shown by simulations, its finite-sample power is, in general, low. Although our explicit results pertain to a simple linear regression, they can be easily generalized. Our empirical results indicate that noncausality is quite common among economic variables, making these problems highly relevant.