Real-time contingency analysis (RTCA) is paramount for modern power systems as it forms the basis for important operator actions that help to improve system stability, optimize generator dispatch, manage disparate resources, prevent cascading outages, and enhance market operations. With increasing system size and the number of contingency scenarios, RTCA is faced with computational challenges. To alleviate this situation, massively parallel graphics processing units (GPUs) are introduced for the acceleration of RTCA solution in this paper, where the compensation method (CM) is utilized for the concurrent AC power flow solution. Strategies and principles on the data structure, kernel function, and memory management are provided. Five benchmark systems (ranging from 300- to 13,659-bus) are employed for case studies. Based on the sequential CM implemented on single-thread CPU, the performance analysis related to execution time and speedup is carried out for parallel CMs running on other architectures, including multi-thread CPU, single-GPU, and multi-GPUs. Results indicate that the parallel CM with multi-GPUs has sufficient accuracy, convergence, and scalability. Finally, the potential of the proposal for practical RTCA has been discussed with the reviewing of other state-of-the-art parallel computing methods reported in the literature.