The paper considers the Riemann zeta function defined as \(\zeta(z)= \sum^\infty_{n=1} n^{-z}\), where \(z= \sigma+ it\), \(\sigma> 1\) and \(t\in R\). A. Ya. Khinchin (1938) proved that for every \(\sigma> 1\), the normalized function \(f_\sigma(t)= \zeta(\sigma+ it)/\zeta(\sigma)\), \(t\in R\), is an infinitely divisible characteristic function. The corresponding distribution function of \(f_\sigma\) is denoted by \(F_\sigma\) and called the Riemann zeta distribution with parameter \(\sigma\). The aim of the present article is to investigate further the fundamental properties of \(F_\sigma\), including its support and unimodality. The main results may be summarized as follows: (1) \(F_\sigma\) is a discrete distribution with support \(\{-\log n\}^\infty_{n=1}\); (2) the jump of \(F_\sigma\) at the point \(-\log n\) is \(n^{-\sigma}/\zeta(\sigma)\); (3) the Riemann zeta random variable can be represented as a linear function of infinitely many independent geometric random variables; (4) the Dirichlet-type characteristic functions are constructed and compared with the Pólya-type characteristic functions of absolutely continuous distributions; (5) in order to extend the Khinchin (1938) result, the Dirichlet-type characteristic function is proved to be infinitely divisible if the coefficient (as an arithmetic function) in the Dirichlet series is completely multiplicative; (6) applying these results, the authors provide probabilistic proofs for some identities that connect the Riemann zeta function with the Mangoldt function, and with the Jordan totient function; (7) two inequalities for \(\zeta\) are also derived.