The QR decomposition is one of the fundamental matrix decompositions in data mining. A particularly challenging case of QR decomposition is to deal with the tall-and-skinny matrix. Tall-skinny QR has lots of applications such as Krylov subspace methods and some subspace projection methods. Furthermore, tall-skinny QR can accelerate the process of principal component analysis (PCA). Although algorithms like TSQR and Cholesky QR have been proposed for computing QR decompositions on tall-and-skinny matrices, none of these algorithms are suitable for being applied to the GPGPU, which has been increasingly used nowadays. In view of the limited memory in GPGPU and also the costly data transmission between CPU and GPGPU, we propose a novel R-initiated TSQR to make the computing of tall-and-skinny QR on the GPGPU efficient. Explicitly, our method is unique in that it utilizes Givens QR to take advantage of the existence of dual-triangular (DT) structure in submatrices in TSQR so as to significantly reduce the computation required. With the R-initiated method, our method can not only meet the memory limitation of GPGPU but also avoid large amounts of data transmission. Theoretical results are derived, showing the merit of the proposed method. The experimental results indicate that our method significantly outperforms the conventional TSQR.