In this paper we continue the study of important Banach spaces of slice hyperholomorphic functions on the quaternionic unit ball by investigating the BMO- and VMO-spaces of slice hyperholomorphic functions. We discuss in particular conformal invariance and a refined characterization of these spaces in terms of Carleson measures. Finally we show the relations with the Bloch and Dirichlet space and the duality relation with the Hardy space $\mathcal{H}^1(\mathbb{D})$. The importance of these spaces in the classical theory is well known. It is therefore worthwhile to study their slice hyperholomorphic counterparts, in particular because slice hyperholomorphic functions were found to have several applications in operator theory and Schur analysis.