The random intercept cross-lagged panel model (RI-CLPM) is an extension of the traditional cross-lagged panel model (CLPM) that allows controlling for stable trait factors when estimating cross-lagged effects. It has been argued that the RI-CLPM more appropriately accounts for trait-like, time-invariant stability of many psychological constructs and that it should be preferred over the CLPM when at least three waves of measurement are available. The basic idea of the RI-CLPM is to decompose longitudinal associations between two constructs into stable between-person associations and temporal within-person dynamics. The present article critically examines the RI-CLPM from a causal inference perspective. Using formal analysis and simulated data, we show that the RI-CLPM has limited potential to control for unobserved stable confounder variables when estimating cross-lagged effects. The CLPM with additional lag-2 effects sufficiently controls for delayed effects, as long as all relevant covariates are measured. Furthermore, we clarify that, in general, the RI-CLPM targets a different causal estimand than the CLPM. Whereas the cross-lagged effect in the CLPM targets the effect of increasing the exposure by one unit, the within-person cross-lagged effect in the RI-CLPM provides an estimate of the effect of increasing the exposure by one unit around the person mean. We argue that this within-person causal effect is typically less relevant for testing causal hypotheses with longitudinal data because it only captures temporary fluctuations around the individual person means and ignores the potential effects of causes that explain differences between persons.