Abstract Background. Electroencephalogram (EEG) microstate analysis can capture transient patterns of brain activity and provide valuable insights into brain motor and cognitive functions. However, the performance of traditional microstate analysis algorithms limits a deeper understanding of the neural mechanisms behind complex conditions. Methods. This study proposed a Canopy-KM-GSA algorithm, which combines Canopy clustering algorithm, K-means algorithm and genetic simulated annealing framework to automatically determine the optimal number of microstates and refine the clustering sequence. Utilizing the proposed algorithm, the study performed microstate analysis of pedaling motor datasets, Passive Auditory Oddball Paradigm task datasets, and epileptic patients datasets. The performance of the proposed algorithm is compared with seven baseline algorithms (including traditional K-means algorithm, K-medoids algorithm, ICA algorithm, PCA algorithm, GMD driven density canopy K-means algorithm, modified K-means algorithm and Agglomerative Hierarchical Clustering(AAHC) algorithm). Results. The results demonstrated the superior performance of Canopy-KM-GSA, achieving a significantly higher total evaluation compared to baseline microstate analysis algorithms. With an average Global Explained Variance (GEV) of 94.43%, an average Calinski-Harabasz Index (CHI) of 537.99, and an average Davies-Bouldin Index (DBI) of 1.57 in pedaling motor datasets; an average GEV of 94.46%, an average CHI of 389.29, and an average DBI of 1.44 in Passive Auditory Oddball Paradigm task datasets; an average GEV of 58.40%, an average CHI of 254.11, and an average DBI of 1.53 in epileptic patients datasets. Conclusions. The novel microstate analysis algorithms offers a more accurate tool for EEG microstate analysis.