Recent studies hypothesized that phospholipids stabilize two voltage-sensing arginine residues of certain voltage-gated potassium channels in activated conformations. It remains unclear how lipids directly affect these channels. Here, by examining the conformations of the KvAP in different lipids, we showed that without voltage change, the voltage-sensor domains switched from the activated to the resting state when their surrounding lipids were changed from phospholipids to nonphospholipids. Such lipid-determined conformational change was coupled to the ion-conducting pore, suggesting that parallel to voltage gating, the channel is gated by its annular lipids. Our measurements recognized that the energetic cost of lipid-dependent gating approaches that of voltage gating, but kinetically it appears much slower. Our data support that a channel and its surrounding lipids together constitute a functional unit, and natural nonphospholipids such as cholesterol should exert strong effects on voltage-gated channels. Our first observation of lipid-dependent gating may have general implications to other membrane proteins.