Frequency response analysis is an important computational tool to simulate and understand the dynamic behavior of structures. However, for more target frequency and/or larger scale structures, the runtime is greatly increased. Furthermore, increasingly complex degree of freedom problems intended to improve the accuracy of the analysis results is creating longer. In this paper, we present efficient analysis using runtime reduction in frequency response analysis with NVIDIA GPU using the compute unified device architecture (CUDA) programming environment. The proposed method is based on the sparse conjugate gradient method and a Jacobi preconditioner. Numerical examples which implemented by three different FE model are used to verify the validity. The results show that GPU parallel implementation achieves significant speed up compared to a single CPU processor. Through these results, in the frequency response analysis, we show the possibility for efficient analysis with reduction of the solving time by using GPU parallel implementation.