As a kind of widely used switchgear in power system, the reliability of gas insulated switchgear (GIS) is very important for the safe operation of power systems. However, there is a lack of research on intelligent detection technology of mechanical state of GIS at present. A new method is urgently needed to improve the operability, effectiveness, and accuracy of fault detection in GIS. Aiming at the abnormal vibration signals generated by GIS faults, this article presents a fault diagnosis method (GA-DBSCAN) consisting of a feature selection method based on genetic algorithm (GA) and Density-Based Spatial Clustering of Applications with Noise (DBSCAN), and a fault diagnosis method based on DBSCAN. First, this article analyzes the incentive force of GIS and discusses the characteristic frequency of response signal combining with the non-linear characteristics of a GIS system. Second, GA and DBSCAN are used to screen features for dimension reduction and get the optimized feature space, and DBSCAN-based classification is used to classify faults. Finally, optimized feature space is verified to be superior to the original feature space by typical classification method; the superiority and reliability of DBSCAN-based classification method under optimized feature space is verified by comparing with other classification methods. The proposed GA-DBSCAN approach can substantially increase the performance of the fault diagnosis method, which indicates that the method promotes development of intelligent detection technology of mechanical state in GIS.