Dunkl operators are differential-difference operators associated with a finite reflection group, acting on some Euclidean space, and they can be regarded as a generalization of partial derivatives and play a major role in the theory of quantum many-body systems. This paper is a systematic work connected with investigation of some probabilistic aspects of the theory of Dunkl operators. The authors give a clear introduction to the Dunkl theory with basic facts on reflection groups, root systems, multiplicity functions; then the associated Dunkl operators, the Dunkl kernel (as a generalization of the exponential function), and the Dunkl transform (as a generalization of the Fourier transform) are introduced. A generalization of a one-parameter semigroup of Markov kernels on \(R^{N}\) is Dunkl's Laplacian, and is called the \(k\)-Gaussian semigroup. Also, the concept of \(k\)-invariant Markov kernels on \(R^{N}\) is introduced using the algebraic connections between \(k\)-Gaussian semigroups and the Dunkl transform. It allows to define semigroups of \(k\)-invariant Markov kernels as well as the associated Markov processes, which are called \(k\)-invariant. A characterization of \(k\)-invariant Markov processes on \(R^{N}\) are unique solutions of martingale problems in the sense of Stroock and Varadhan. Some limit theorems for \(k\)-invariant Markov processes are considered. Namely, a law of the iterated logarithm for \(k\)-Gaussian processes, a strong law of large numbers for general \(k\)-invariant processes in discrete time, and a transience criterion. A generalization of Ornstein-Uhlenbeck processes to the Dunkl setting is given. Also, the authors systematically study modified moments of higher order for \(k\)-Gaussian measures, which are connected with a generalization of Hermite polynomials and the Appell systems.