Abstract In this paper, a study of the channel modulation instability of commercial p-GaN gate HEMTs is presented. During the gate-voltage stress test, substantial RDS(ON) variations up to 78 mΩ (93.8%) were observed. It is found that the p-GaN/AlGaN/GaN gate structure enables the injection of holes and electrons, which can be captured by the donor/acceptor-like traps located in the AlGaN layer. Therefore, the trapped holes and electrons concurrently modulate the channel conductivity, resulting in RDS(ON) variations. Device simulation was performed to help explain the mechanism from the perspective of energy band. In addition, results reveal that with the recommended working gate-voltage stress VGS = 7 V, the on-state resistance, the threshold voltage and the off-state drain to source leakage current vary up to 8 mΩ (16.3%), 0.2 V (14.8%) and 12.8 μA (42.66%) within 1 h, respectively, which could raise reliability issues for the power electronics applications of p-GaN gate HEMTs.