A mediation effect explains the relationship of a risk factor and an outcome through a mediator variable which is a step in their pathway. Under the assumption of no cycling in the causal relationship, we consider various situations in which a fourth variable may interfere the estimation of a mediation effect as a confounding factor. Our asymptotic results, which are supported by a Monte Carlo study, show that adjusting for confounding factors under certain conditions might lead to biased estimates. A general guideline is provided for when it is appropriate to adjust for confounding factors in estimating a mediation effect. We apply the guideline to the estimation of the mediation effect of Alzheimer's disease pathology in the relationship between the Apolipoprotein E ε4 allele and cognitive function among 125 deceased participants from the Religious Orders Study, a longitudinal, clinical-pathologic study of aging and Alzheimer's disease.