The unitary Cayley graph on $n$ vertices, $X_n$, has vertex set ${\Bbb Z}/{n\Bbb Z}$, and two vertices $a$ and $b$ are connected by an edge if and only if they differ by a multiplicative unit modulo $n$, i.e. ${\rm gcd}(a-b,n) = 1$. A $k$-regular graph $X$ is Ramanujan if and only if $\lambda(X) \leq 2\sqrt{k-1}$ where $\lambda(X)$ is the second largest absolute value of the eigenvalues of the adjacency matrix of $X$. We obtain a complete characterization of the cases in which the unitary Cayley graph $X_n$ is a Ramanujan graph.