It is shown by way of a simplified model that the calculation of the time nonlocal potential resulting from an energy-dependent optical potential is not unique. Most choices result in a noncausal time-dependent potential, in spite of the inclusion of appropriate scattering boundary conditions in the model. Similarly, potentials arising from the many-body treatment of the optical potential are typically noncausal. However, it is demonstrated that, because of the energy spectrum of the Hamiltonian, the appropriate wave equations for scattering and the wave-function solutions in both approaches are nevertheless causal.