What are the constraints placed on the constitutive tensors of elastodynamics by the requirements that the linear elastodynamic system under consideration be both causal (effects succeed causes) and passive (system does not produce energy)? The analogous question has been tackled in other areas but in the case of elastodynamics its treatment is complicated by the higher order tensorial nature of its constitutive relations. In this paper, we clarify the effect of these constraints on highly general forms of the elastodynamic constitutive relations. We show that the satisfaction of passivity (and causality) directly requires that the hermitian parts of the transforms (Fourier and Laplace) of the time derivatives of the constitutive tensors be positive semi-definite. Additionally, the conditions require that the non-hermitian parts of the Fourier transforms of the constitutive tensors be positive semi-definite for positive values of frequency. When major symmetries are assumed these definiteness relations apply simply to the real and imaginary parts of the relevant tensors. For diagonal and one-dimensional problems, these positive semi-definiteness relationships reduce to simple inequality relations over the real and imaginary parts, as they should. Finally, we extend the results to highly general constitutive relations which include the Willis inhomogeneous relations as a special case.