It is well known that the solutions of delay differential and implicit and explicit neutral delay differential equations (NDDEs) may have discontinuous derivatives, but it has not been sufficiently appreciated that the solutions of NDDEs -- and, therefore, solutions of delay differential algebraic equations -- need not to be continuous. Numerical codes for solving differential equations, with or without retarded arguments, are generally based on the assumption that a solution is continuous. In this paper, the authors illustrate and explain how the discontinuities arise, and present methods to deal with these problems computationally. These methods include discontinuity tracking; perturbing the initial function; direct treatment of NDDEs in Hale's form; singular perturbation methods for NDDEs in Hale's form. The investigation of a simple example is followed by a discussion of more general NDDEs and further mathematical details.