Abstract In this paper, a novel output-only structural system identification method is proposed based on the decomposition of the response signal by applying newly developed symplectic geometry mode decomposition (SGMD). The SGMD can decompose the output signal into a limited of single component signals which are also called symplectic geometry components (SGCs) adaptively without determining too many parameters. Then the random decrement technique (RDT) is applied to extract the free decay responses from SGCs. Finally, the modal parameters such as modal frequency and damping ratio can be identified using the Hilbert transform (HT) method. To demonstrate the validity and effectiveness of the proposed method, a numerical example and a monitored quayside container crane structure are conducted. The modal parameters of those studies can be obtained accurately and effectively by using the proposed method.