The paper is a continuation of the research works of the authors in connection with solving of Toeplitz systems (or Yule-Walker type systems) by the preconditioned conjugate gradient method using circulant preconditioners. The authors propose some circulant preconditioners with the aid of the spectral density function of the given discrete-time stationary process. Using these circulant preconditioners for solving a Yule-Walker type system by the conjugate gradient method, superlinear convergence is obtained. In the case of unknown statistics, a sample with a finite number of data of a random process is used for the estimation of the autocovariances, as a least squares problem, by using four different windowing methods: (W1) the correlation method, (W2) the covariance method, (W3) the post- windowed method, (W4) the pre-windowed method. With these estimations the circulantly preconditioned matrix, necessary for solving the Toeplitz system, is derived. In this case, the convergence of the iterative process is considered in a probabilistic way. Among the results of the paper we mention Theorem 2 (p. 367), that shows that if the discrete-time stationary process satisfies a set of assumptions, then the derived circulant preconditioner is positive definite and the least eigenvalue of this matrix is uniformly bounded away from zero, with probability 1, if the sample is sufficiently large. In the end of the paper, numerical experiments are performed to test the convergence performance of the algorithm. These numerical results show the effectiveness of the algorithm for least squares problems.